Russ Tedrake [00:00:00] Machine learning success empirically has gotten far ahead of our ability to understand it theoretically. I feel like we've had to change from being engineers where we design everything on first principles to we're becoming more scientists, almost like behavioral scientists, where we're building things we don't fully understand, and now we have to go probe them to figure out what the heck just happened.

Brian Heater [00:00:19] Things are changing in such fundamental ways that it must be both an exciting and very challenging time to be running a physical AI startup.

Russ Tedrake [00:00:27] Totally. But one has to be focused enough to not chase after every shiny bauble.

Brian Heater [00:00:34] How do you determine when something is a shiny object and when something is actually a real game changer?

Russ Tedrake [00:00:42] That's a good question. So I'm spending a lot of time these days talking to labor economists and with people that I think could be potentially impacted and just trying to build my empathy muscles and build my understanding. I do think if we're successful, the field is successful, then it's gonna change the very nature of work for people, and it's gonna have a hopefully hugely positive impact on the world.

Brian Heater [00:01:04] I'm sure that there have been plenty of points in your career where you've been approached to do something like this, and I'm sure that you've thought about it in the past.

Russ Tedrake [00:01:12] I will tell you there's one reason that's maybe not even technical that I really wanna do this. First of all, I think--

Brian Heater [00:01:31] Hello, and welcome to another episode of Automated. My name is Brian Heater. I am the managing editor at the Association for Advancing Automation. Here's a great conversation from our marathon of Boston shows from a few weeks back. Russ Tedrake is somebody that we've been looking to get on the show for a while.

I figured that we would have to wait until he officially had something to announce, but it turns out that showing up at his doorstep also worked. More info on that new project soon, but here is some insight into his work in the space to hold you over. If you're enjoying the show, don't forget to like and subscribe, and please check out the newsletter over at automated.fm.

And with that, please enjoy this chat with Russ Tedrake.

Russ Tedrake [00:02:16] You grew up in Detroit. I grew up just outside of Detroit.

Brian Heater [00:02:19] Your father was in automotive?

Russ Tedrake [00:02:21] That's true. General Motors.

Brian Heater [00:02:23] So you were, in a sense, kind of in and around automotive manufacturing automation.

Russ Tedrake [00:02:28] Yeah. My dad actually collected classic cars. We had a drive-out basement, and we had a '21 Packard touring car and a '48 Cadillac, which was the very first of the tailfin. But I learned a lot about cars back in the day, yeah.

Brian Heater [00:02:42] Tell me a little bit about the internship that you had at the Ford plant.

Russ Tedrake [00:02:46] So when I was in high school, a lot of the big car makers had a lot of small contract shops around town. There was one that my neighbor ran that did paint shop improvement. So they went into paint shops in the local plants, and they would optimize the airflow and other things about the paint process. I took an internship there over the summer, so I got to write some automation that tried to control the airflow through the plants.

I was the junior guy on the totem pole, so when somebody had to climb into the sludge pit with the sensor, that was my job the first time. But I did work up to writing some of the initial code that would sort of automate the systems. The most defining part of that experience for me was the day that my--I thought--clever code, which when there was an exception, like somebody tripping over an ethernet cable, I thought the safe thing to do would be to turn off the air fans. Air speed go to zero. But that was the wrong thing to do because the temperature in the booth went from, I don't know what it was, 81 to 82 degrees, which was the unionized threshold where people walked off the line because they didn't have to work at 82 degrees, and I got yelled at.

I really learned a hard lesson that day about what it means to stop the line in an automotive plant. But I grew up a lot that summer.

Brian Heater [00:04:11] So yeah, the lesson is really knowing the product that you're creating and knowing all the consequences.

Russ Tedrake [00:04:19] I was just a kid writing some code, but it was awesome to see the plant in action. That was in the Ford Wayne Assembly Plant.

Brian Heater [00:04:26] It's wild that they entrusted you with something like that.

Russ Tedrake [00:04:28] There were people watching me. But I did screw that one up myself, yeah.

Brian Heater [00:04:32] And then you were also studying video games at a certain point?

Russ Tedrake [00:04:37] That's true. So when I was at University of Michigan--go Blue, they just won the national championship last night--the robotics program was kind of in flux there. Dan Koditschek had actually just been leaving right around then. John Laird was studying AI there. I worked with him, and the closest thing to robotics I could get into at the time was doing AI for video games. So I got to spend time with John Laird programming AI for video games. I spent a summer at Microsoft Research doing AI for video games.

Yeah, that was my entryway.

Brian Heater [00:05:14] It sounds, in hindsight, like a pretty good entryway, especially looking at the way NVIDIA and everything played out as far as simulation and physics and robotics.

Russ Tedrake [00:05:25] For sure. Yeah, it worked out. I learned a lot. Also just being at Microsoft at that time was really a fantastic experience. I think the company has grown and changed a lot since then, but I really enjoyed those summers.

Brian Heater [00:05:38] How did you get into bipeds?

Russ Tedrake [00:05:41] So when I first came to MIT, Gill Pratt was just running the Leg Lab, but he was leaving, so he didn't take on new students at the time. But I got to hang out with Jerry Pratt and Dan Paluska and a few other people. I got to hang out in the basement of NE43 where the Leg Lab was. Pete Dilworth was a guy there at the time.

He built Troody the dinosaur robot, which to my young eyes was just like, 'That's what I wanna do with my life.' So I hung out with Pete and with Dan, and I worked on M2 and I worked on Troody a little bit. And I fell in love with that.

So I had to find my own path through that, and the path I took was through passive dynamic walkers--walkers that were inspired by simple toys that could waddle down a ramp, but showed that the dynamics of walking was beautiful. And so that was my... My thesis was a little bit more on the dynamics of walking.

Brian Heater [00:06:09] WobbleWorks.

Brian Heater [00:06:45] Showed that the dynamics of walking was beautiful--but something that could actually be integrated into robotics at a certain point?

Russ Tedrake [00:06:55] If you think about what robotics was at the time, we had a lot of robots that were walking very conservatively, so like a stiff astronaut kind of--

Brian Heater [00:07:04] Yeah, I think ASIMO comes to mind.

Russ Tedrake [00:07:05] ASIMO was there. ASIMO was actually beautiful, but even before that we had a lot of really stiff walkers, really slow. Mark Raibert had built some hopping robots that were throwing themselves through the air, which was extremely inspiring.

And then there was this kind of outlier--a guy named Tad McGeer, and then Andy Ruina at Cornell also followed up wonderfully on this work--who built these robots that you put on the top of a ramp, you give them a little push, and some of them had incredibly human-like gaits. Way more human-like than what we were seeing from the actuated robots. But they only walked down a ramp, so it seemed a natural milestone to figure out how to add a little bit of actuation. Let physics do most of the work, but add just a little kiss of power and control to make it walk around. And so that was the inspiration.

That was my little thesis robot. I used machine learning--I used reinforcement learning back in the day, before it was cool--to make that work. So I had a little robot that learned how to walk in 20 minutes. That was basically the thesis work.

Brian Heater [00:08:03] So the idea was if we can get it to move down a ramp, then the learnings potentially we can use to get it to walk in other situations.

Russ Tedrake [00:08:15] You got it exactly right. So learning to walk from scratch was considered very hard at the time, and so we basically kept it... it was already walking if the motors were commanding only zero, and then you just slowly lowered the ramp, and it basically kept learning, kept learning, and then just walked right off the end of the ramp.

Brian Heater [00:08:33] As somebody who was steeped in bipeds at the time, are you surprised at all how things have really come around, and come around this quickly, to bipeds and humanoids?

Russ Tedrake [00:08:47] It was always the dream. Am I surprised? I think it's amazing to me that it is as turnkey as it has become. That is awesome. That is success. The hardware is maybe more amazing than I might have dreamed, in terms of the cost and the capabilities that we're seeing. But I always believed. That's why I was doing this.

Brian Heater [00:09:12] But it does seem like maybe we skipped a couple of steps in there, especially when it comes to building useful robots in the field. I mean, we were out there talking to Locus yesterday. And obviously when we talk about really successful robots out there, we think about AMRs. There is a halfway point, or a missing link, between an AMR and a humanoid robot. Are we going after AGI, or going after general purpose maybe a little too quickly right now?

Russ Tedrake [00:09:51] Well, I don't think of walking--the solutions that we have for walking--as being directly on the path to AGI or needed for that. I think there was maybe inevitably, people were living the dream of trying to understand--biology makes this thing like bipedal walking look easy, and our robots are sort of failing at this. So that's a puzzle that needs to be solved, and we've had great minds pushing on that for decades.

So it's great that it's been solved as well as it has been solved. I don't think it's completely solved, but I think it's pretty turnkey at this point. Separately from that, I do think there's something extremely exciting about building more humanoid-like robots and exploring what AI can potentially do with a powerful general-purpose body. I think that's one of the most exciting things happening in the world right now.

Brian Heater [00:10:48] You were talking about being ahead of the curve when it came to machine learning. What was the state of machine learning at the time, and in what ways was it being deployed in robotics?

Russ Tedrake [00:11:00] So people knew about reinforcement learning for a really long time. Rich Sutton and others had been vocal advocates for many, many years. I remember in 2004--that was when I did my thesis--Peter Abbeel had his helicopters that were doing a type of apprenticeship learning, and I had my little toddler--we called it Toddler, the learning robot--that was just using very simple reinforcement learning to learn how to walk.

That was the first time in a handful of years that we had seen RL and learning working on real robots, so that got people kind of excited. But there was still a lot of skepticism. In my class, I used to teach a whole curriculum of robotics, and I had a section--a third of the class was set up for reinforcement learning. And you could tell people were just kind of like, 'Yeah, I don't really care. I don't think this is gonna go the distance. Reinforcement learning's a cute idea, but it's not gonna really work.' And then I actually slowly weeded it out of the curriculum, because people just didn't seem that interested in it.

Brian Heater [00:11:59] Do you get a sense of what the source of the skepticism was at the time?

Russ Tedrake [00:12:03] It wouldn't scale, right? That it was too data inefficient, sample inefficient.

Brian Heater [00:12:07] Okay, so the same things we're kind of talking about right now.

Russ Tedrake [00:12:11] What changed is the amount of compute that we could throw at the problem.

Brian Heater [00:12:15] But the data problem is still a conversation we're having.

Russ Tedrake [00:12:18] The data problem for manipulation is a slightly different one. I think what the capabilities in locomotion were really... a couple of magical things happened.

First of all, what we learned is if you can do the right amount of domain randomization and in simulation take your robot model and walk over stairs and walk over bumps and just a handful of different things, somehow that is good enough to make a robot walk over almost anything in the real world. That was not expected to be that easy. It works incredibly well. It's just a huge result.

Separately, the simulation and GPU infrastructure has made that just absolutely... And it's now open source available. The fact that anybody can do that and that that recipe's kind of out there is just amazing. I also do think that simple policy gradient style reinforcement learning worked with neural networks as the parameterized model--worked better than most people thought.

Brian Heater [00:13:28] I am always fascinated when really smart people who work on these things and know these fields really well use words like 'somehow.'

Brian Heater [00:13:37] It sounds like there was a surprise, and at least at the time you weren't quite able to explain why it worked as well as it did.

Russ Tedrake [00:13:46] That's definitely true. I have used that analogy several times in the last few years--machine learning success empirically has gotten far ahead of our ability to understand it theoretically. And I feel like we've had to change from being engineers where we design everything on first principles, to we're becoming more scientists, almost like behavioral scientists, where we're building things we don't fully understand, and now we have to go probe them to figure out what the heck just happened. Which was a fascinating transition.

Brian Heater [00:14:18] How does that change the nature of your role?

Russ Tedrake [00:14:22] It totally changes that. You have to think more like a scientist that's trying to... Well, there's one view of the world, which is we don't have to understand. Let's just pour in more data, pour in more compute, follow the recipe, and we'll get better and better things out.

Brian Heater [00:14:38] I suspect that's not quite your view.

Russ Tedrake [00:14:41] I think you have to do that also. I do subscribe to that, but I also want to go through and try to understand what the heck just happened. Maybe just for my own psychological wellbeing. Possibly because the field will hit a wall at some point. Who knows? And then our deeper understanding will help us push through when that time comes. But I don't think we've seen the limits of scaling yet. I think we've got more headroom to go.

Brian Heater [00:15:06] Obviously, again, this is coming at it from somebody who is very much not a scientist, who is a liberal arts major. But to me, it seems obvious that you would want to understand the machinations there. You want to understand how they're working so you can recreate them, so you can continue that research going forward.

Russ Tedrake [00:15:29] That is the goal of science, I think. We have an opportunity to understand intelligence--physical intelligence in this case, right? It used to be we'd have to probe Drosophila or find other ways around it. But the fact that we're creating things that are incredibly capable, and we have the code, and we can measure activations, and we can start asking fundamental questions--we can change the data recipe, we can change the learning rates, and start to really study these things in an unprecedented way. So that's a huge opportunity.

Brian Heater [00:16:05] What do you mean when you say 'ask fundamental questions'?

Russ Tedrake [00:16:08] So I think there is just an understanding of like, why does it work at all? Why does something as simple as gradient descent on these super overparameterized models manage to find the things it seems to find? So there's a whole theory of deep learning, which has got incredibly talented people making lots and lots more insights. I'm actually super excited about how that field is progressing.

There's also sort of things that are one step closer to practical for a robotics practitioner, right? Where we want to understand how to efficiently use data. If you want to construct a data curriculum for your robots to learn as much as possible with as little compute as possible, theory can potentially help with that. Certainly, extremely well-organized empirical science can help a lot with that. So understanding robustness, understanding privacy, understanding all these fundamental questions that you want to get answers to.

Brian Heater [00:17:12] It's interesting. I've been really thinking about this in the context of obviously you at TRI and you're in the process of launching this startup. But for a very long time you've also been a professor, you've been in academia, and it strikes me that I don't want to say it's difficult, but it's uncharted territory to be teaching something that you don't fundamentally understand.

Russ Tedrake [00:17:44] That's interesting. But you can still give people--I mean, people teach science. We teach about biology, even though our understanding of it is via probes and experiments and a massed understanding. So I do think maybe we have to teach it slightly differently.

We give people experiments that they can probe. Here's a set of code that can teach a biped to walk--go mess with it. Figure out why it works. Or build it yourself. That's what's so amazing right now.

Brian Heater [00:18:15] So a lot of it is really like, teach me how this thing works.

Russ Tedrake [00:18:20] Yeah. There's a lot of that. I think it's both amazing and probably challenging to be a student in 2026. I think you can do almost anything. The world is at your fingertips. You can start kind of like vibe coding almost anything. But at the same time, things are moving so fast, and how do you pick the right problems and how do you chart your own territory and where is the field gonna be? How do you sort of skate where the puck is going when things are moving this fast and changing so quickly? So I think it's an incredible time for the field.

Brian Heater [00:18:57] That's something I wanted to also talk about in relation to running a startup. I was talking to the CEO of Rhoda AI. And he just offhandedly said, obviously, we're following all of the research that's coming out, and every once in a while something must come out and it must be like, 'Uh-oh, we have to rethink.' So you are sort of maybe in a similar position to the students in that things are happening so quickly and things are changing in such fundamental ways that it also must be both an exciting and very challenging time to be running a physical AI startup.

Russ Tedrake [00:19:37] Totally. But I've been doing it for a while, and I think one has to be focused enough to not chase after every shiny bauble. And I think setting super clear directions and helping students or other folks sort of know what's important, what's worth paying attention to--but also being nimble enough so that when something amazing and new happens, you recognize it as such and are willing to change.

I think that is the skill set now--to be able to walk both of those lines. Keep the steady hand when appropriate, but pivot quickly when appropriate.

Brian Heater [00:20:20] How do you determine when something is a shiny object and when something is actually a real game changer?

Russ Tedrake [00:20:35] Well, I've always tried to be more hands-on, so I write code at night and try to have a... I worry about getting too arm's length away from it. So I really try to dabble a lot myself. That helps a lot. I surround myself with people who are incredibly well-versed in this, and I trust their judgment.

I remember it used to be that students would ask me what papers they should read, and now it's like students are sending me all the papers. 'Wait a second--is this the one I have to read this week?' You've gotta filter a little bit for me. There's a deluge of papers that come at you, and you have to be selective. But I also think over the years you can pretty quickly recognize what the key results are and when they happen. It's an art for sure--I don't know that there's a playbook for it.

Brian Heater [00:21:30] I think it's helpful too when you recognize half the people, and maybe a quarter of them were your students at some point.

Russ Tedrake [00:21:38] That's just saying I'm old, but--

Brian Heater [00:21:40] We were discussing it before, and I think one of the really wonderful things I've noticed in covering robotics is that it's big and getting larger, but there is and always has been a way in which it feels like a really small world, and everybody knows each other. And people largely--and certainly this applies to you--seem to be very supportive of one another.

Russ Tedrake [00:22:03] Yeah. No, it's incredible to see--to have had the opportunity to work with really a bunch of very special people. MIT really gives me the chance to work with some incredible people. To see what they've gone on to do has just been one of the joys of my life--to see what they're capable of, what they're doing, how they're changing the world. There's nothing that makes me prouder.

Brian Heater [00:22:29] So there is a way--it's funny because I'm coming from TechCrunch, and I'm coming from the startup world where everything feels so competitive and cutthroat. But there is a way to sort of coexist in this world, where you can root for other people's breakthroughs and still kind of be a competitor with them.

Russ Tedrake [00:22:51] Yeah. So let's talk about it. I think there's a voice of skepticism in the air, there's a voice of enthusiasm in the air. In my mind, a number of incredible things have happened all at the same time, and it makes me extremely optimistic. I think almost nobody doubts that robotics will change our world dramatically. The biggest question is, when? Is this the time? I think we have a bunch of energy, a bunch of momentum, certainly from GPT and people getting excited about what AI is capable of.

Brian Heater [00:23:32] Is this time somehow fundamentally different than the other times when it kind of felt like this?

Russ Tedrake [00:23:36] Exactly. The reason I think it's different is certainly some incredible things have happened in the technology, but it's more than that. It's that that has led to an influx of talent--the number of incredible people who have come into the field from other disciplines that are working on robotics or physical AI now. The amount of investment that has come in. The changes in the supply chain. It's incredible what's happened in China in terms of the ability to manufacture these very capable devices at cost.

The world needs this technology in certain ways. The changing expectations for work, the silvering society--there are so many things that have aligned to make me think the field has an incredible opportunity. I feel like we're shooting for the moon. Do we have escape velocity? I think so. I think this is the time it's really different. And I would rather be on the rocket ship.

Brian Heater [00:24:46] You describe a phenomenon that's really interesting to me. We were just speaking with Clara Vu right before this, and she was saying that when they [founded] Veo, at the time a lot of the people they were getting were either coming over from automotive or self-driving cars, or medical devices.

And something I've noticed in robotics is that as a discipline, when it comes to bringing people in, it doesn't seem to be particularly rigid. That's kind of amazing--I don't see quite the same thing in other fields. As long as they're smart people, you will accept them and find a way to integrate them into the system.

Russ Tedrake [00:25:31] I do think we draw from many professions, and many people who did not do robotics before are certainly welcome. I also think Boston is an incredible place to be building a robotics company. The ecosystem here with MassRobotics and the number of incredible robotics companies here--it's an incredible place to be doing that.

I do think we draw on many different disciplines. People who weren't previously excited about robotics see what might be coming and want to be involved. That's awesome. That's one of the reasons I'm excited--we do have incredible people coming into the space.

Brian Heater [00:26:10] You talked a little bit before about coding every night to kind of keep your skin in the game, and I'm curious--at TRI and now in your new role--what is it that you're really doing most of the time?

Russ Tedrake [00:26:27] It depends on the day. Maybe not this month. This particular month has been--

Brian Heater [00:26:34] So just to back up--you told me that yesterday you just moved into a new space.

Russ Tedrake [00:26:39] Yeah. Well, let's just take teaching, for example. So when I teach, I try to provide a lot of examples, worked examples. I make demonstrations for class, which involve building up an intense sort of big pipeline of software engineering to get students to the point where they can very quickly experiment with an idea, or show an idea on a slider or anything like that. That is really where all the components come together into the open course notes.

Behind that, I've been working hard on this simulation and model-based design software called Drake. I kind of joke that it's my Horcrux--I put a piece of my soul in that software package. I still contribute code into production for Drake, and that's important to me. It keeps my skills sharp. It keeps me thinking the way I want to be thinking.

Brian Heater [00:27:34] So TRI--that was a big part of your role at TRI, Drake. And then obviously you're the large behavior model guy.

Russ Tedrake [00:27:42] Something like that. Gil Pratt came up with the name.

Brian Heater [00:27:45] But it's in the company name.

Russ Tedrake [00:27:52] Yeah. No, we're super proud of that work. It changed my view of what's possible in the field. I think TRI had an incredible role to play there--trying to do the science of LBMs at a level that wouldn't immediately be motivated to be done in a startup, but might not also be accessible given the resources in academia.

So I think there was a particular role that TRI could play on the science of LBMs--understanding the initial scaling laws, but doing it with a lot of experiments that were required to dig in and understand rigorously what those scaling laws could look like. Very proud of that work.

Brian Heater [00:28:37] Would you have called it something different?

Russ Tedrake [00:28:38] No, I thought it was a great name. I like to try to clarify for people the difference between a VLA and an LBM. Large behavior models in my mind are any model that takes sequences of images in and outputs actions. So we're training large multimodal models. A VLA in my mind is a particular type of LBM.

One choice for building large behavior models is to uptrain a VLM--a vision and language model--into being a robot model. That's one architectural choice you can make. If you make that choice, I would say you've built a VLA. But there are other choices. You could start with a video backbone or a world model backbone, and uptrain that to be an LBM.

So I just like LBM as kind of--from LLMs to large behavior models--as a natural way to talk about the space.

Brian Heater [00:29:43] So it sounds like, at TRI and maybe the work that you're doing, it's kind of that large umbrella approach to training?

Russ Tedrake [00:29:53] Yeah. The methods--you have to be willing to try lots of different architectures, although the architectures have been more converging. There's still lots of room for improvement there. But I think we kind of know now that if you want longer context lengths, you should be starting with a video model as a backbone. Multimodal models are getting better and better, so you should absolutely set yourself up to be able to leverage the advances that the field is making there.

But I think having a code base with some maturity that you can quickly take in the latest model is really important in this time.

Brian Heater [00:30:32] I was reading up a bit on Generalist before this, and I'm sure that you've been following the work that they've been doing. One of the things that they were saying is that they sort of scrapped their own model and started from scratch. Is that something that you feel is necessary versus starting on top of an LLM?

Russ Tedrake [00:30:54] So there's this whole narrative about data paucity in robotics--that we don't have enough data in robotics. And it's interesting to see different teams have different answers to that. They say, 'Okay, well, maybe I'm gonna learn everything from egocentric data,' or, 'I'm gonna learn everything from simulation,' or, 'I'm gonna learn everything from the universal manipulation simple interfaces or from teleoperation.' I think that whole narrative misses--there's something I would articulate a little bit differently.

Brian Heater [00:31:38] This is the flywheel we're talking about, right?

Russ Tedrake [00:31:42] Yeah. I think that story misses a bigger picture, which is that in robotics we're starting with a strong model and uptraining it to be a robot model. So the data you need for robotics is not every piece of data ever to teach a robot common sense about the world.

I think we're building on models like video models that already have a lot of common sense about the world, and the data you need is to build a bridge from that common sense--to basically train the model to have one extra output, which is robot actions. Which is just a little bit different than what people are saying, and it positions you to go after architectures and data a little bit differently if you embrace that.

So absolutely, you should be using simple grippers, you should be using egocentric data, you should be using simulation, but you should be thinking about that as building a bridge from the common sense that's already in the model. I mean, if I took a picture of my robot in a real application and gave it to the latest Gemini video model, Veo 3, let's say, and gave it a text prompt saying, 'Make a video of my robot doing some dexterous task,' it's gonna do a pretty good job, right? So that suggests there's already a lot of understanding in that model. What you have to do is teach it how to map that output--instead of making pictures--to map it into robot trajectories. And that's a very different posture.

So back to Generalist--I would not throw away the base model. The value of that base model is so great. People are spending immense resources compiling massive datasets to build these base models that have so much world knowledge. I would not throw that away. I would start from there, and then think about your data curriculum as building the bridge to teach that model how to act on your robot. I think Generalist is doing that at a high level. I think we're doing that at a high level.

Brian Heater [00:33:50] One of the things that I did appreciate that they said--and I suspect maybe you're on a similar wavelength, especially as we're talking about large behavior models being this kind of umbrella term--is that they're talking about being method agnostic. And the people that I've spoken with who make the most sense are the ones who say all of these will have their purpose for pre-training, for post-training, even teleop when it comes to post-training. Is this effectively where you stand?

Russ Tedrake [00:34:34] Yeah, I think--I've been doing it with a super strong team for enough years that we've tried all of them.

They all have their place. Being very thoughtful about pulling them in, understanding what they're good for--I like the analogy of building bridges. So if you have simulation data of a particular KUKA robot doing a particular task, then maybe you also want video of a KUKA, and that allows you to take your video dataset and your simulation dataset and connect them together. And that allows these models to transfer capabilities more effectively.

Kitchen sink approach to data, but then filtering very thoughtfully.

Brian Heater [00:35:13] Are we anticipating that there's going to be another one of those surprise moments when it comes to dealing with the edge cases or the long tail when actually putting these robots out in the world?

Russ Tedrake [00:35:25] That's the point. I think we've seen the beginnings of that. We saw that in the LBM paper that we published--we saw the beginnings of that where you saw robustness in individual tasks change in a substantive way by having done pre-training on other tasks. I think that is the magic of multitask pre-training.

Brian Heater [00:35:47] And that--if I'm remembering the work correctly--is that the diffusion policy work?

Russ Tedrake [00:35:55] Diffusion policy was the original idea. Large behavior models was the multitask version of diffusion policy, in my vernacular.

Brian Heater [00:36:03] So these are still tasks in controlled environments. As far as more generalization and moving outside of the TRI offices--factories and warehouses--where does that magic come in?

Russ Tedrake [00:36:32] This is, I think, the next major milestone for the field--to get robots deployed in the world. I think the narrative of many people has shifted towards deployment. We have to earn that. The field has to earn that. And if we get that, then we do get this virtuous cycle where more capable robots makes more robots in the field, makes more data, which makes more capable robots, and I think we get a real virtuous cycle going there.

For me, I'm very excited now about taking that next step of really deploying these things in the real world and understanding the right way to do that.

Brian Heater [00:37:08] So there is a data gap from the standpoint of obviously there just are not enough systems out in the world collecting that real-world data. That is a real issue.

Russ Tedrake [00:37:19] Yeah. Data makes things better. But I only want to rephrase some of the discussion that says we don't have as much data as a language model--that's not the way I think about it, because we're starting with the language model. We're starting with the base model that has already been trained on the internet. And we're adding to that.

That just reframes what you think you need from the data--what you really need is to map that into the tasks you care most about or the customers care most about.

Brian Heater [00:37:48] So here's where we sort of get into the sticky territory for you, but insofar as you can talk about this, what role do you see yourself and your company playing in that world?

Russ Tedrake [00:38:01] I do think we're on escape velocity. I want to be on that rocket ship. I don't think it's inevitable. I think it takes incredibly good execution with an incredibly strong team and a lot of focus--not chasing every shiny bauble. And it takes a good business plan. You have to do a lot of things right to make this work.

But I do think the component technologies, the investment in the field, the talent coming into the space--everything has aligned to give us a real chance. The company that I'm excited to announce soon is built to basically maximize the chances of us trying to get to the next level.

Brian Heater [00:38:53] I'm curious--and again, this is a bit of an abstract question--as the team is coming together and as you're pitching yourself as a team versus other physical AI companies out there, obviously a big selling point is the talent you've brought on, but beyond that, is it IP or what are the special sauces you're bringing?

Russ Tedrake [00:39:23] I think there are a lot of things that go into making us unique--from the way we think about data, the way we think about deployments, the way we think about operations, the way we think about our business. So there are a lot of things you have to get right. But I am pretty proud of the team. It's an awesome team.

Brian Heater [00:39:46] You feel though that there are a lot of angles you're looking at that differ quite a bit from the way that--

Russ Tedrake [00:39:55] Yeah, for sure.

Brian Heater [00:39:56] I'm sure that there have been plenty of points in your work and your career where you've been approached to do something like this, where you've been approached to launch something, and I'm sure that you've thought about it in the past.

Russ Tedrake [00:40:07] That's a good question. I have been asked many times why would we want to do something new--why not join an existing effort? I will tell you there's one reason that's maybe not even technical that I really wanna do this. First of all, I think there's a particular--we're building a team and a business--to have a chance of success.

I do think if we're successful, the field is successful, then it's gonna change the very nature of work for people, and it's gonna have a hopefully hugely positive impact on the world. But it is gonna be a big change for people. Working with Toyota over the years had really shaped this notion of amplifying, not replacing people--thinking about what are the implications for the future of factory work, how do we help people age in place more gracefully.

It has really shaped the way I think about this, and I'm spending a lot of time these days talking to labor economists and spending time with people who I think could be potentially impacted, just trying to build my empathy muscles and build my understanding--both the macroeconomic understanding of it and the in-the-small understanding of how it could impact people. And I think the companies that are most thoughtful about this have an opportunity to make even small changes as they go forward and really help people get the best version of this.

Brian Heater [00:41:49] That's a very good answer, and something I have not heard discussed a lot. To break it down a little bit--the Toyota piece is there. Obviously, aging in place is a big part of Toyota because it's been such a fundamental piece of Japanese culture in general. It's an aging population. That's part of the reason why robotics have been big over there. And then as far as amplifying people--I think I was talking to Max, actually at TRI when we were over there, and I think what he was saying is he was talking about how deliberate the factories are and set up. Everything is there for a reason, so if you're gonna bring in a piece of automation, it better be there for a reason.

Russ Tedrake [00:42:41] Yeah. I think there are wonderful things about Japanese culture. Lifetime employment is also a strong theme traditionally in Japanese culture, and so the narrative from TRI is real. There's a real need to do this correctly.

Brian Heater [00:42:58] Again, big broad question, but what does it mean to approach something like robotics or physical AI with empathy?

Russ Tedrake [00:43:08] I think--let's see how we've already seen it play out, for maybe many people in your audience who have experienced the software engineering version of this or maybe the graphic design version of this. We've seen the field of software engineering change, right? People who are already expert software engineers are now superpowered by Claude Code or whatever your favorite tool is. And people who are already incredible artists can use Photoshop or NanoBanana or whatever to be able to do their exceptional art very quickly. Novices can vibe code almost anything now. And I can make logos, I can make art in a way that I certainly couldn't a couple of years ago.

So what is that gonna look like in the physical world? I think we're gonna have a lot of tasks that would have been very hard to automate before that are suddenly vibe codable almost. We'll have top craftspeople who could do magnificent things, more capable, more quickly, with super tools.

But how does that change a culture that distributes wealth through labor--where people's sense of worth, sense of purpose, often comes through the job they've taken? I think we have to think very carefully about that. You just have to spend time helping people understand what the tools are capable of, helping them adjust, using the best parts of what they can do and amplifying that with proper tools. I think we have to solve this for artists, we have to solve this for software engineers, and we're gonna have to solve it for physical labor too.

Brian Heater [00:45:04] And this is a role that a physical AI company could ultimately play.

Russ Tedrake [00:45:09] I think a thoughtful physical AI company could change the way that plays out.

Brian Heater [00:45:13] You biked over here. The shot of you on your academic page looks like you're standing on top of a mountain. Does the time that you spend out in the natural world impact the way that you interact with technology?

Russ Tedrake [00:45:36] That's interesting. I do super value my commute. That sounds like a weird thing to say. But I run or bike to work, and it takes time, but that is my time to decompress, to compile everything that happened through the day, to think more holistic thoughts. There are famous thinkers through the ages that took long walks. That's been defining.

Brian Heater [00:46:13] Yeah, Thoreau was a big walker.

Russ Tedrake [00:46:15] Thoreau went for four-hour walks every day and then wrote for four hours a day. It's a nice way to think more deeply. It's so easy to just answer emails all day long. I love my commute.

Brian Heater [00:46:32] So you can vibe code, but sometimes you've gotta shut it off and step away.

Russ Tedrake [00:46:37] I think so.

Brian Heater [00:46:40] Well, Russ, thank you so much for taking the time.

Russ Tedrake [00:46:42] Thank you for your questions. I really appreciate it.

Brian Heater [00:46:43] Thanks to Russ and the whole team at MassRobotics for hosting us in Boston all week. Thanks as always for tuning in. Please like and subscribe and rate the show. Check out the Automated newsletter that hits inboxes on Thursdays and LinkedIn on Friday. And with all of that, we will see you next week for another episode of Automated.