[00:00:00] Daniela Rus: It's not so difficult to get a car to steer on an empty road. The big challenges have to do with dealing with the messiness of the world. If your car is driving at 60 miles an hour, does the car have the luxury to wait for the cloud to tell it what to do? I would say no. We are at such an extraordinary moment in computing, in robotics, in AI.

I just wanna live every second of it and, and make, the most contribution I can.

[00:00:28] Brian Heater: You were pretty down on humanoids or at least bipedal humanoids. Is that... Have you softened at all?

[00:00:33] Daniela Rus: There are many advantages that, there are, for the humanoid shape, but we are not ready for prime time with humanoids just yet.

[00:00:42] Brian Heater: I've spoken with a lot of, physical AI companies over the last several months and, and I feel like everyone is again moving towards these really large models for robotics. Is, is that the, the direction ultimately that you see things going in when we're talking about embodied AI or physical AI?

[00:00:58] Daniela Rus: We are at a point in time where it's important to explore a lot of different directions, and so we, we see university research, we see startups exploring in many different directions.

My own view is that-

[00:01:25] Brian Heater: Hello, and welcome to another episode of Automated. My name is Brian Heater. I am the managing editor at the Association for Advancing Automation. We are back with our second ever live show, and one of my favorite people in the whole of robotics. Daniela Rus is the director of MIT CSAIL, among, many other things.

Chatting with her always offers some incredible insight into just how many world-changing aspects are currently being impacted by robotics. self-driving cars, we've got decoding sperm whales. This conversation has it all. Thank you so much to Daniela and her team, and of course MassRobotics for hosting us.

If you're enjoying the show, don't forget to like and subscribe, and please check out the newsletter over at Automated.fm. And with that, please enjoy this conversation with Daniela Rus. You know, we talk a lot about what's coming up next in automation on this show, but if you really wanna see the future in motion, you've got to be there in person.

Automate 2026 is where the world's leading innovators, builders, and dreamers come together to show you what's possible. Robots, AI, machine vision, motion control, you name it, all automation under one roof. And as part of Automate this year, the Humanoid Robot Forum brings together leaders, engineers, and researchers for a two-day deep dive into the real-world development, deployment, and commercialization of humanoid robotics.

Register for free at automateshow.com to join us in Chicago June 22nd through the 25th. We will see you there. The last time that I saw you- We were, I think we were at the Automate show, and it was very brief, and I think I was chasing you through the halls. You had your suitcase with you because you had to jump on a plane.

Thank you for doing this. I feel like I'm a busy person. I don't know how you possibly maintain your schedule at all.

[00:03:20] Daniela Rus: Brian, just like you and everyone else does. So yeah, sometimes I have to run through hallways with a suitcase.

[00:03:27] Brian Heater: Yeah. Yeah. I mean, obviously you've, you've got a lot going on. You've got your teaching schedule.

You're on a number of boards. It seems like you're, you're flying all over the place. I mean, how do you determine what you prioritize at any one given time?

[00:03:40] Daniela Rus: You know, honestly, I just wanna do it all. Yeah. I feel like we are at such an extraordinary moment, in our field and, in computing, in robotics, in AI, and I just wanna live every second of it and, and make, the most contribution I can.

It's, I think we're very lucky. We're really the generation that is bringing, intelligence and, robots and automation, to the world in a way that supports people and supports the planet, and, and it's an extraordinary time.

[00:04:14] Brian Heater: Yeah, 'cause you, you and I have been speaking for a, a number of years now, like, probably close to 10 at this point, and when we first, first started speaking, there was a lot of, you know, five to 10 years, five to 10 years, but it really feels like the last few years, a lot of these things you've been talking about have really started coming to fruition.

[00:04:31] Daniela Rus: Well, yes. So 10 years ago, Brian, you were the only one who wanted to talk with me. That can't be true. It was really great. but, look, technology development is like this. you make, steps. You have, you have people who think about the future, who think about what might happen 10 years from now, 25 years from now.

And, at MIT CSAIL, we have always been focused on what can we do today that still has relevance and impact 25 years from today, and we really have been thinking about living in the future. the exciting thing is that the future we were dreaming about 10 years ago is now, and now we have to kind of figure out how do we think about the future 10 years from now, but also be in the moment, be in the now, because, there is so much excitement and opportunity.

And you see universities have the mission of advancing the knowledge, of showing that ideas, can, can take flight, that capabilities are possible. This is what we do at the university, and there is this extraordinary, attraction to thinking about what we used to think about back then, and, and, and to see that is, to see it happening right now.

And I would say that it's not trivial to take a research-grade solution and turn it into a solution that is impacting the world today into a product. And so that part of the innovation cycle is also exciting and, and important. And, I, I, I'm just trying to figure out how I can do some of that, and I can continue to do the things that I've loved doing all my life.

[00:06:20] Brian Heater: I wanna take two pieces of technology that, that I know, CSAIL has been involved in and ha- obviously have taken two very different paths. There's, autonomous driving, and then there are LLMs. In terms of where you expected things would be and where things are, it seems like self-driving has maybe taken a little bit longer.

LLMs seem to have kind of surprised everybody. You know, even people who worked on them seem surprised at, like, how quickly they got as good as they got.

[00:06:48] Daniela Rus: Yeah, so it's exciting. self-driving, cars have been, and, and in some sense navigating robots, have been, part of the conversation in the robotics community for decades.

In our research group, we had... Our first, self-driving research vehicle was navigating the streets of Singapore in 2011, 15 years ago. It's taken some, research demonstrations and then companies like Google Waymo and others, to take those ideas and make them robust. The, the challenge is the long tail, right?

I mean, it's not so difficult to get a car to steer on an empty road. what, what is challenging-

[00:07:35] Daniela Rus: But, in today's, in today's, landscape, that is not, that, that is not where the big challenge is. Yeah. the big challenges, ha- are, have to do with dealing with the messiness of the world, dealing, with people who break the traffic rules, dealing with weather. So there are a lot of edge cases, and what we call the long tail. And, and so we continue to work on, on these ideas. Sensors don't work well in snow and in rain. That's why we don't have, rides by Waymo in Boston. Hmm. Boston is too chaotic. Actually, even the way people drive in Boston makes it really difficult, for a robot car that's expecting people to follow the rules to operate effectively.

But, we continue to make progress, and we already see a lot of, interesting capabilities from the self-driving, industry. We see, of course, the cars that are offering autonomous rides to people, but then there are a lot of other powerful applications that the world is talking about less. For example, we have autonomous driving in industrial settings that is m- making a huge difference.

Our company, Venti Technologies, has been, deploying fleets of autonomous 40-foot-long trucks that are supporting the operations in the Port of Singapore, and they've been operating 24/7 for months, for more than a year now, and they will be deployed in other settings. there are other companies that are built, that are bringing these self-driving solutions, in industrial settings, in, ca- on campuses, in, in, areas that are off public roads, and the technologies are really good for those kinds of environments.

The technology is really good for, either, closed environments, which are very carefully mapped, or for environments that are simple, where things don't move very fast, where we don't have so much uncertainty from the perception system, of the car. So great progress in self-driving vehicles, but still we have a long way to go.

[00:09:51] Brian Heater: Also, just so you know, I, I like to point out that probably nothing that we're talking about in robotics right now would be really possible in the way it is without all of that progress that had been done with self-driving cars.

[00:10:01] Daniela Rus: Well, exactly. Exactly. We, I mean, you know, this, this progress in robotics is enabled by advancements in hardware, by advancements in computing, and by advancements in algorithms.

And for autonomous driving, the key contributions, had to do with making maps, with perception, with de- detecting things reliably so that you don't bump into things, the car doesn't bump into things. also with planning and control. But the hardware plays a very important role. so having accurate, maps, really depended on having, laser scanners, the LiDAR sensors, and then figuring out how to couple the feedback from a LiDAR sensor to the feedback from a camera.

You know, back in the day when I was a graduate student, I used to, build navigation algorithms using sonar, and sonar is extraordinarily, noisy, and, nothing worked back then. After the LiDAR was introduced, all the algorithms that didn't work before with sonar, all of a sudden they all started working because measuring distance with light is much more precise than measuring distance with sonar.

And so the hardware that is available to us is very important, in, solving the problem. So, the sen- sensors and actuators are important, but also the speed of computation, is important. And we will continue to see great progress moving in the future. For autonomous driving, what's difficult now is, driving in inclement weather.

So what do you do when, when it rains really hard? What do you do when it snows and you don't see the road? And at MIT, we've been looking into this use case in this part of the, self-driving problem, and we, we have a solution, that is not, deployed in, commercial products yet. But it turns out that if you, put a ground-penetrating, radar under the car and you look at the texture of the soil, we see unique features that could serve, as landmarks, just like the features, we collect with LiDAR for locating inside the map, do.

So, like, it's possible to deal with weather by looking down rather than looking up. Isn't this interesting? And this is a capability that is enabled by a hardware advancement coupled with a software advancement.

[00:12:39] Brian Heater: I, I'm, like, want to back up a second 'cause I'm curious about the, the sonar thing. It's just always been really interesting to me, You know, in cases where you're working on a technology like that and, and really kind of i- in a certain sense, you know, banging your head against the wall, and then something a couple of years comes along and it makes it a lot easier, can, can that be frustrating that you were, you know, that it was this really difficult research and then...

No? No,

[00:13:03] Daniela Rus: No, it's exciting, right? It's exciting. Yeah. Like, you want, you want the work to, to have an impact. Yeah. And so, the algorithm, the b- the basis of the invention remains the algorithm. The algorithm couldn't work when, the, the sensors were imprecise, but all of a sudden we got a precise sensor and boom, everything started, working.

That was an extraordinary moment.

[00:13:26] Brian Heater: Yeah. You, you had a, a moment, it was, it was John Hopcroft, right? That was the visiting Iowa professor, and it sounds like not long after you had moved to the States, he kind of changed everything for you as far as, your career path.

[00:13:43] Daniela Rus: Oh, I still remember that day. So I, I went to a lecture that was given by one of the founding fathers of the field of computer science, John Hopcroft.

And, interestingly in this lecture, he said, "Computer science is dead." And, well, what he meant was a lot of the problems that were posed by, by the, early founders of the field had solutions. So he actually didn't mean it... Maybe he didn't say computer science is dead. Maybe he said computer science is solved, and now it's time for the grand applications, and the grand applications were about computation that interacts with the world.

And in, in some sense, this is what robotics is. And so I was very excited by this vision of building machines that interact with the physical world, and I went to work with him for, for this objective.

[00:14:35] Brian Heater: What was it about him specifically that made you really... I mean, you f- you followed him. You went and worked with him.

I mean, he really did change that path for you.

[00:14:42] Daniela Rus: He is a visionary. Yeah. He really sees into the future, with great clarity. And back in the day when I was a graduate student, he had the vision of robots. He also had the vision of, simulation environments where, you can test how machines work, and that could be shared by labs across the world.

There were... The, networking was not good enough to do it back then, but look where we are now. All the predictions he made back then have come to pass.

[00:15:16] Brian Heater: Yeah. So that, that sounds like a, a familiar story then. A l- a lot of times, there's just sort of a bottleneck. You know, there's a lot of these kind of ideas behind the scenes, but the technology just isn't there to execute them.

[00:15:29] Daniela Rus: Well, the technology and the ideas, that's why it's important to follow the path and build on everyone's prior work.

[00:15:37] Brian Heater: What do you see your role being at a place like CSAIL?

[00:15:41] Daniela Rus: Well, I, as, the director of CSAIL, I see my role as thinking about the future, and building programs, and bringing ideas for my colleagues to help develop.

I'm very interested in topics that are bigger than an, an individual researcher's, area of expertise. Mm. So I've been looking at, well, what are some of the, the, challenges and, and problems that require expertise from m- multiple aspects of computing, maybe even reaching out to other disciplines, and how can we then create an opportunity for people to come together and address those challenges?

So from an intellectual point of view, I'm very focused on new ideas that will build the future 10 years from now, 25 year- years from now.

[00:16:37] Brian Heater: Yeah, I, I, a very long time ago now, I, I think I sat in on, on one of those, those meetings. it was sort of almost like a pitch meeting it, it felt like. how, how do those projects really sort of start to, to spin out?

[00:16:51] Daniela Rus: It has to be a kind of a combination of top-down and bottom-up conversations. So top-down and bottom-up because, if the leader of, of an organization like MIT says, "Let's work on X," well, that path is not successful unless people are interested in working on X. And so, or at the same time, if the group decides that Y is interesting, but there's no way of identifying the resources, to enable Y to proceed, again, we're stuck.

So we ha- we have to, combine the, the ability to identify resources for a project with the people's interests to work on those problems.

[00:17:38] Brian Heater: So let, let's take a recent example. This is something, again, that you and I spoke about a number of years ago. You were very excited. You said, "We can't talk about this yet."

And then I looked in The New York Times not too long ago, and it seems like you're finally really ready to talk about it, a lot of the research that you had been connected with around, sperm whale breeding. That news has finally surfaced. MIT CSAIL was involved in that. What was your role, and, and how did you get involved with that?

[00:18:08] Daniela Rus: I've been interested in, whales and, how technology can help us understand, their lives for more than a decade. And that all came to be because I was at the MacArthur Fellows gathering meeting, and I heard Roger Payne, give a, a lecture about his beloved whales, and I was just mesmerized. And so I still remember going to Roger to say, "Hey, I love your, your whale song.

What can I do for you? Can I make you some robots? Is there something that, you need that would help you, study the whales better?" And so we... So back then, this was 2007, 2008- Oh, wow ... 2009, it was really a long time ago. back then, my students were building drones, and so we thought, "Oh, maybe we can take some drones, to observe whales close up."

And we did indeed, have a project where we took drones, and we studied, whales off the coast of Argentina, in 2009. And the kind of, behaviors and footage that, we could, get with the drones was, was extraordinary. Roger spent years on a cliff with paper and pencil and binoculars, essentially looking at who goes by, and he knew who went by because whales have callosities on their heads, southern right whales do, and these callosities are unique to every individual.

And so for about 40 years, Roger stood, stood on a cliff and, and kept track of who he could see went by. So now imagine the change that, that he experienced when he could essentially just throw a drone, out at sea to really look close by. Well, who are they? What are they doing? the kind of, of, data and resolution that he could get with a drone was, something that he didn't even, imagine, and Roger was quite the visionary.

So it was transformational for his work. And then a few years later, we were at the workshop at Harvard, and we started talking about, well, you know, what if we could, exchange, with whales. What if we could someday understand what whales are saying to each other? What is, what is part of the, the communication mechanism, the vocalization, and the song of whales?

So we wrote a, proposal for a TED Audacious Project to reverse engineer the language of sperm whales, and we were awarded the prize. A, a lot of people were involved, in this program. Roger certainly was. I was part of it. David Gruber, Rob Wood, and others were all part of writing this proposal. We got the funding, and we've been, studying the, recordings of, the sperm whales, in Dominica. The sperm whales, click, so they just go something like that. And it's interesting because their language is discrete, and as a result, it's much easier to analyze using computer science techniques. And, we've had several, breakthroughs in, in our understanding of, of the language of, sperm whales, but we're still quite far from really knowing what they say and, what is the right way to think about, this, these insights and innovation.

We've, analyzed the, the alphabet, and we've shown, using AI, we were able to show that in fact there are many more, units of speech, which are called codas, than previously, imagined. We showed that, whales have the possibility of expressing about 540 different types of, sounds, and prior to our work, it was believed that they could only do 21, 20 or 21.

And so this actually, matters a lot because when you can make, a lot of sounds, that means you have the capacity to transmit and encode a lot of complex information, and this is what makes the difference between, animals that can vocalize, simple things and animals that can transmit, complex ideas.

[00:22:57] Brian Heater: I mean, I feel like I've had a lot of conversations at cocktail parties where it's just like, "Hey, let's, you know, figure out if we can get whales to talk," but you're actually able to, like, execute on that, which is, that's a nice part of your job.

[00:23:07] Daniela Rus: Well, it's so exciting. Yeah. I told you, it's, it's an extraordinary time.

So, look, we first have to understand- the vocalization and the communication structure, and then we have to understand how that connects to their behaviors and their lives, and then we have to understand what do we do with that so that we do not endanger the whales, but rather we protect them.

[00:23:34] Brian Heater: Yeah, I, I think that that's an underrated probably part of, of your job, certainly part of my job, and maybe just part of, you know, robotics, r- robotics in general is, a curiosity and an interest in, in other fields.

I mean, that's sort of what, what unites us. That's a big part of what you do is, you know, being interested in whales, being interested in, in sea turtles, in landmarks, and that's what all of a sudden opens up all of these other different fields, and that's what connects all these dots for you.

[00:24:05] Daniela Rus: There are definitely great opportunities across disciplines, and computing and robotics and AI have matured to the point where, we can connect these tools to other disciplines to enable discovery in other disciplines.

[00:24:21] Brian Heater: Yeah. W- we t- I mentioned, l- alluded to, LLMs a little bit before, and, and I think you were one of the first people I spoke to who was really talking about the black box problem. and when I talked about this sort of, surprise that researchers had around, a lot of these large language models, it was because of the black box, right?

You know, all this information was going in. They didn't really know why the results were coming out. and one of the things that you have been advocating for, and I think really still are through Liquid AI are a lot of these, these smaller, these smaller models. Why is that something that you have been so focused on?

[00:24:58] Daniela Rus: Well, Brian, the extraordinary pace of progress in AI is really inspiring, and you have to understand that many of the ideas that are enabling these extraordinary capabilities are ideas that were invented decades ago. Hmm. And back then, we didn't have enough data, and we didn't have strong enough computation.

We didn't have enough memory. And so back d- back then, they didn't work so well. But, n- then we had an inflection point where the hardware platforms and the s- and the data sources, got to the point where all of a sudden those ideas started working, and that was extraordinary. In fact, I still remember, the talk that Sebastien Bubeck did at MIT in March of 2023, where he could reveal, some of the capabilities of GPT-4. We, the world did not know that GPT-4 existed, and, he showed extraordinary programming capabilities. He showed extraordinary language generation capabilities, reasoning. It was stunning. it was absolutely stunning. And since then, of course, the field, to make progress and the capabilities increased.

And right now, what we see with agentic AI is, is unbelievable. The ability of, the gen- the, the tools to generate code and to give us organized access to information is absolutely extraordinary. What's interesting is that now we have these huge models, that require industrial scale installations and, that have a lot of benefit, but you, you sort of need deep pockets in order to take full advantage of the capabilities in the cloud.

And so I, I see this moment as being analogous to where computing was, back in the day when we had mainframe computers, and only companies with very deep pockets were able to, take advantage of computing. And then forward-looking researchers, some at Lincoln Lab, Bill Gates, Steve Jobs, their friends, they invented the personal computer, and the personal computer completely democratized access to computing and created an economy that we didn't imagine could exist.

A lot of our everyday comforts and, and the way of life today is due to computing. Just imagine a day in your life without computing and everything it enables. No email, no social media. Maybe no email is good. I mean, no social media, no diagnosis imaging, no electronic commerce, right? Like it's, it's impossible to think about it, today.

So I am very excited about the possibility of bringing AI from the cloud, down to earth. there are many devices, on our planet. In fact, a large fraction of processors are in phones, in, iPads, in, computers, in, sensors. They're in everyday devices. So the question is, can we empower those devices, with the ability to reason with, with a kind of AI that offers the maximum level of intelligence according to the hardware specifications of the device without the need of a cloud? I personally believe in a hybrid future, where there continues to be a need of large scale, operations in the cloud because ultimately, the, the current technology really depends on how much information you can put inside the models. So the larger the model, the more information you can package, in that model. But not all problems require access to all the world's information. So what if we actually had, customized models that run on your phone, run on your computer, run on your sensors, and on your robots, and that run and deliver the capabilities that you need without the need to talk to the cloud, without the energy cost, without the water cost, that is required in order to bring large AI to the world?

This is what, I have been, thinking about for many years now, and, you know, honestly, this was originally motivated by robotic problems because we wanted the models to run on device. If your car is driving at sixty miles an hour, does the car have the luxury to wait for the cloud to tell it what to do?

I would say no, but if you have a, a well-tuned model, that runs on device, that understands physics, that understands, common sense in the context of driving, then you can have the AI reasoning capability on device, and you can have much, safer driving. When we started working with Toyota in two thousand and fifteen, we had this vision of the car that will never be responsible for a collision and become your friend.

So now as we think about the state of the art in driving, we've made a lot of pro- progress on the autonomous car that, that would not be responsible for a collision. The friend part was a little bit slower- ... to come, to come to fruition, but with large language models, the car can become your friend by putting the language inside the car.

[00:30:55] Brian Heater: I, I've spoken with a lot of, physical AI companies, over the last several months and, and I feel like they're going down a similar path where, you know, as we're talking about sort of general- generalized AI, you know, g- general purpose robots, that everyone is, again, moving towards these really large models for robotics.

Is, is that the, the direction ultimately that you see things going in when we're talking about embodied AI or physical AI?

[00:31:21] Daniela Rus: We are at a point in time where it's important to explore a lot of different, directions, and so we, we see, university research, we see startups exploring in many different directions.

M- my own view is that for AI to really deliver, good capabilities for robots, safe capabilities for robots, we need those AI systems to run on device. We need those AI systems to understand physics, and we need those, AI systems to essentially understand the task. And robotic tasks are prim- primarily about the interaction between the machine and the physical world.

Mm-hmm. And interaction requires knowledge of forces and torques. So the kind of, physical AI that we are developing in my research group at MIT, aims to capture that. And the forces and torques are really difficult to capture if your robot, learns how to do things by watching videos or even by teleoperation.

So, I think it's interesting to consider what kind of tasks, can be taught by purely visual feedback, what kind of ta- tasks can be taught by teleoperation, and for what kind of tasks do you need, more detailed information like forces and torques? And when we think about robots and, and whether AI is an important part of the robot solution, I think it's important to consider whether the task of the robot, is something that can be described from first principles.

Mm. Because if that is the case, then we don't need complex models to guess, correctly what to do next. but, we can, we can deliver a solution from for, for, from first principles. However, if the task is so complex that it cannot be described from first principles, then we need to figure out how to collect the data to enable the machine to learn.

And, at MIT, we're looking at complex tasks, that are done by humans that are difficult to, map onto, onto equations. And, and so what we do is we instrument people doing, let's say, kitchen tasks or construction tasks. We instrument people, and we collect, information about the muscles, so forces and torques.

We also collect pose information. we collect, eye gaze information. We collect environmental information. And so from all of this data, we are then able to synthesize policies that teach machines how to do the task in a manner similar to what the human did.

[00:34:23] Brian Heater: Yeah. the conversation that, that I've been having a lot lately with a lot of these people is, is the flywheel problem, right?

Is, is how do we actually get enough of these robots out into the, the world? or do you get the sense that it's ultimately going to require, Physical, physical real world data will a certain, how much of this will actually be able to be executed with, like, synthetic data and simulation, for example?

Or do the robots actually have to be out there performing tasks in order to really get a good idea of physics and long tail problems?

[00:35:01] Daniela Rus: I think it depends on the task.

[00:35:03] Daniela Rus: And so I believe that there are some tasks that can be, can be trained from purely visual data. We don't know exactly what those tasks are, but the research community is working on, on those kinds of policies.

But then there are also tasks that I believe require understanding of the world, understanding of physics. And so for these tasks, what should we do? Well, one option is to try to collect more physical data. Another option is to think about different kinds of AI models that have a, a notion of physics that are baked into them that are alternatives to the transformer models that are being, broadly, distributed. And, then there are other ideas where maybe you can take what your transformer-based model, produced as a potential solution, and then you can simulate it to see if the solution, matches the constraints and the physics of the tasks. and so I, a- we don't have a, a clear path forward.

This is a very interesting point in time, where we are exploring all these different avenues, and eventually we will have something really amazing.

[00:36:21] Brian Heater: Yeah. One, one of the things that's been really interesting to me the more I've been thinking about this is, is when we talk about embodied AI or physical AI is, you know, when we tend to think of our bodies, we tend to think of sort of a central brain, you know, moving all of our different parts, but physical AI, it could be, you know, like a per sensor, a per actuator basis.

[00:36:41] Daniela Rus: What is important to remember about robots is that they have a body and a brain.

[00:36:46] Daniela Rus: And so we need to think about what we want the body to do and, and create the morphology that is capable of the task, and then we need to have a brain that can control the body to do what it's meant to do. And these two pieces are interrelated.

We can, we can put the brain anywhere. We can put the brain, as a, as a central coordination mechanism. We can also create decentralized systems, where the brain is spread across the, the robot body. And interestingly, there are organisms in nature that operate according to very interesting distributed intelligence, like the octopus, for example.

So in my- a research lab, we have a new project, where we are looking at how do we create intelligence in a decentralized way, in a way that is inspired by the way the octopus distributes, its neurons across the entire body. How do we get machines to learn skills? How do we get machines to, to compose those skills?

And this is actually, an important detail because most, solutions that we have today for robots learn skill in context. So that is not a very general solution for skills. So are there new ideas that enable the learning of skill that can be then applied, to any kind of context, through some form of adaptation?

And then also, how, how would a machine that has this kind of sophisticated ability to, to adapt and learn new things, how does that machine learn to do new things? So, like, imagine you have a robot with one arm, and then all of a sudden you add a second arm to the robot. What can the robot do, right? So there are certain things you can do with one arm, but when you have two arms, then you can have, extraordinary richness of things you could do that, that one arm, is insufficient for.

So there are some really profound questions related to, learning when we have decentralized intelligence in the machine.

[00:39:10] Brian Heater: Yeah, and this brings to mind other, biologically inspired work that your team and other teams have done as far as, like, swarm-based robots. So w- when we're talking about decentralized intelligence, not just on the robot itself, but just, like, decentralized across different robots.

[00:39:27] Daniela Rus: Well, exactly. So I've been very interested in decentralized intelligence for decades, really thinking about collaboration, how do robots work together to do more together than what a robot can do alone. And, it's interesting to think, collect- about collectives of robots. back in the day, I, I started wondering whether we could imagine self-organizing robot systems where we had, robotic unit modules that could connect with other modules and, realize sort of cellular, organisms, and the cellular organisms, could change.

They could grow a second arm if the task required a second arm, or they could, grow a very tall neck if the robot had to go look up, f- whether a tool is on the, on the top of a shelf, right? So, so there's, there has been a lot of, exciting work, in this, in this area of decentralized cellular robotics.

And, interestingly, the biggest challenge here is the hardware platform. Mm. So we have a lot of algorithms that allow us to reason about what a group of cells can do together. And, when it comes to realizing those capabilities, at, at the hardware level, it becomes tricky because, because the, the components that we have right now are really, largish.

So you cannot make a small, unit module. And then if you have a large unit module, even, even its behaviors are challenging, because making and breaking connections and, doing so robustly is difficult. So there's still a lot of work that we have to do to understand decentralized intelligence when the cells are connected to each other.

We have made a lot of progress on decentralized intelligence, where the units are connected by Wi-Fi, by communication. And so, the swarm, work that, you hear about most often is really about this kind of, this kind of work, a group of ground robots doing together or coordinating with a group of flying robots to get better access to information and better situation awareness.

[00:41:54] Brian Heater: I, I interview a lot of, a lot of artists and, and musicians and creative people, and a lot of the, the... a lot of times we talk about constraints and the power of cons- cons- constraints when it comes to actually sort of, like, focusing your creativity. And I think that there's a parallel to be drawn here, something that, that I hadn't really realized, and, and I hope this is accurate, but something I hadn't really realized about some of your work is part of the reason why you had started doing 3D printing was kind of out of necessity, because that's what you had access to.

You know, I'm wondering when it comes to robotic hardware, whether in your history there has been power in those constraints, in not having necessarily always had access to sort of the latest and greatest and most expensive hardware.

[00:42:40] Daniela Rus: I've always said engineering is a very creative discipline, because the, the f- the real constraints we have to do our work, force us to think outside of the box.

So I started 3D printing, w- in, my first year, as assistant professor. And so- That would've been around when? Eh, let's not go there. But- 2024,

[00:43:06] Daniela Rus: It, it seems, it doesn't seem that long ago, honestly. But I, I took this job at Dartmouth, and I was in the computer science department where I had students who were brilliant programmers, but they did not know how to make stuff.

And so all of a sudden, the 3D printer, the fused deposition, machine came, online. I spent my entire startup package to buy this machine, and then we started making, robots through programming. We started making components that were colorful. The opportunities, opened up to be creative about how we design the robot, so that the robot is not just bunch of metal parts that are stuck together, but, there could be a sense of aesthetics about the body of the robot.

So, that was very, very interesting and exciting.

[00:43:57] Brian Heater: Y- y- years ago when we were, when we were first really starting to talk about generative AI and, and I'd asked you, you know, what applications you were excited about as far as generative AI and robotics, you were really excited about design and, and, you know, the ability of generative AI to actually design robots, which I've actually s- seen some research recently come out from your lab on that front.

[00:44:18] Daniela Rus: Well, we are in the process of writing a paper about an, an, an extraordinarily exciting system that could be the beginning of an AI engineer, where the idea is that you could specify the function of something that you want to build in language, and then you could have a, a generative AI system, maybe a language-based system, study the world knowledge to understand what are the functions that you, you need for a device, that meets your task specification.

So for instance, let's say you want to operate a syringe. You want to create a custom robot, that automates a lab and needs to operate the syringe. Well, that robot has to hold the syringe, and it has to operate the plunger, and so these are the degrees of freedom that ultimately have to be mapped to mechanisms.

And so then our system is able to reason through, through these degrees of freedom and create a mechanism that realizes that, task under the specifications that we give. So this is actually quite exceptional because now you don't have to program a 3D printer, to make a mechanism that, you have to design bottom up with great detail.

What you do is you tell your AI engineer, "Hey, I need a robot to operate the syringe," and the AI engineer will, crank through the world's knowledge, will identify the, the primitives, will create mechanisms, for those primitives, will then feed this information to a simulator to see if indeed the design meets the specifications, and it will optimize.

It will repeat the loop and optimize. So in a matter of hours, you could, in principle, generate what used to take people, months and sometimes even years to get right.

[00:46:18] Brian Heater: So, so to really, boil that down to the, the basic, like, you know, a newspaper headline, it's like ChatGPT for robots, like build, build me a, build me a pizza robot?

[00:46:30] Daniela Rus: It's not just ChatGPT. Right? I mean, ChatGPT is... Or let's just call it the chatbot. Yeah. Let's not show pref, That's fair ... preference for one of the- Okay ... models. So let's just say the chatbot, is essential as an interface, as a way of mapping a concept or a high-level idea, which is what humans are used to, deal with, into constraints and mathematics that machines can deal with, and then from those constraints and mathematics, you could have another AI system that can generate the solutions for those constraints.

[00:47:10] Brian Heater: Well, we're, at about time right now, but I, but I have to ask you, y- you did an interview last year- You were pretty down on humanoids, or at least bipedal humanoids. Is that... Ha- ha- have you softened at all?

[00:47:23] Daniela Rus: I think humanoids are extraordinarily exciting. We have a robot lab at, MIT CSAIL. We call it the Living Lab.

You may have

[00:47:32] Brian Heater: Called them overhyped. Maybe that's the- Where,

[00:47:34] Daniela Rus: Where we have created, a, an apartment, and we have humanoid robots, and a lot of students who are, thinking about how to make those humanoid robots more capable. I think it's very easy to get a humanoid to, delight us, with an extraordinary video, but then, getting the humanoid to work robustly and, and systematically, on a wide range of tasks is complex. It's complex because humanoids are complex mechanisms that are difficult to control, and we still have to figure out the, the technical aspects that enable the humanoids to interact with the world in a dynamical way, so not just statically where you move very slowly or you only move your upper, body, but, but dynamically.

And we still have to think about how the humanoids will understand the constraints of the world and common sense when they interact with us in the world. I was at a meeting a few months ago, and there was a demonstration of a humanoid. It did pretty well, albeit slowly, at watering a plant, and, it interacted with me with...

By voice. And after the humanoid watered the plant, I said, "Now can you water my friend here?" And the humanoid was ready to- ... dump water on the expensive Italian shoes of my friend. So this is the kind of, common sense that, humans would never, mistake, make a mistake. But for robots, this kind of common sense reasoning has to be built as part of the robot function.

So it's one thing to put a humanoid robot in a constrained manufacturing, setting. It's a different constraint, it's a different story to put that robot in a human-centered environment. So I think it's important to con- continue to push, the boundaries, to, develop humanoids. There are many advantages that, there are, for the humanoid shape.

But we are not ready for prime time with humanoids just yet. I think it's gonna take a while longer to get, humanoids to be controllable, in the way we imagine, and to be interactive in the world in the way we imagine.

[00:50:01] Brian Heater: Well, Daniela, always a pleasure. Thank you so much. Thank you. Thanks to Daniela and the whole team at MassRobotics for hosting us.

Thanks as always for tuning in. Please like and subscribe. Please rate the show. Check out the newsletter, that hits inboxes on Thursdays and LinkedIn on Friday. And with that, we will see you next week with another episode of Automated.