How Robots Fail, Learn, and Improve: A Conversation with Holly Yanco

Transcript

Holly Yanco (00:00)

The use cases for telepresence robots where I think it could be really valuable is possibilities for allowing people with disabilities to travel to museums through telepresence robots. So if they couldn't get there, they could be there. I think they're good use cases. I know Vigo back in the day was working with letting students go to school using them. So if students were immunocompromised, they could use them. So I think there are cases for them, but I think we got so used to using Zoom and other video conferencing systems that...you know, do we need to have a presence somewhere else?

Brian Heater (00:42)

Welcome to Automated. My name is Brian Heater. I am managing editor at A3. We have another robotics legend on the show this week. You think I would be sick of saying that by now. Among a lot of other things, Holly Yanko is the distinguished university professor of computer science at the University of Massachusetts. Lowell. Buckle up because you are going to learn a lot from this one. Thanks to Professor Yanko for chatting with us.

Thanks to you for tuning in. Please like and subscribe and follow the automated newsletter over at automated.fm. Here's Professor Yanko. Professor Yanko, thank you for joining us and congratulations on the upcoming move. Thank you. We're recording this in September, one of the first comments I saw on your LinkedIn page is what's going to happen with all the PEZ dispensers. And that must be a huge logistical part of moving from one city to another.

Holly Yanco (01:26)

Thanks. Yeah, there's quite a few of them in my office. Robots, of course, need to move, but ? yes, I have thousands of Pez dispensers in my office.

Brian Heater (01:48)

Tell me a little bit about how you accrued those over the years.

Holly Yanco (01:51)

So this collection started, and here's some of them, back in 1993, ? a group of us from grad school had gone away for spring break and we went into a store and they had Tweety Bird Pez dispensers, which at the time, if you were watching Seinfeld, there was an episode of Tweety Bird Pez dispensers. Somebody bought them for all of us and it ended up on my desk and then people started bringing them to me. So I went from having one to ten to a hundred pretty quickly and now nobody can bring them to me because they don't know what I have so I have to buy all the new ones myself but I've collected them over the last 30 years and I think when students come to my office they're perhaps a little overwhelmed but I think it's fun to have them. It makes the office very colorful.

Brian Heater (02:32)

It's nice to have a thing. I went to school at UC Santa Cruz and visited a professor who has students from all over the world, travel all over the world and bring wind chimes back. So he's just got like a room full of wind chimes. And I was watching a presentation that you were giving on teleop and you mentioned that one of the robots had a Hawaiian shirt and we're debating whether or not that was a marketing first or a robot thing first. It's always good in academia to have some sort of, I don't know, some kind of calling card.

Holly Yanco (03:01)

Yeah, and actually, when my PhD students graduate, you may know that at Brown, when the graphics students graduate, they get a rubber chicken. ? So I decided it'd be nice to have something to give my students. So I actually give them a giant robot PEZ dispenser. So instead of dispensing individual PEZ, it dispenses packs of PEZ, but it's a big robot PEZ dispenser.

Brian Heater (03:19)

Yeah, and I promise we'll move on from the background after this, but I have to ask you how you got your hands on a gold KeepOn.

Holly Yanco (03:25)

It was actually painted. Wendy Ju had painted them for HRI 2012. So they were awards for the video competition for HRI. And I was one of the general chairs that year. So she had made some extras. And she didn't want to take them home with her. I ended up with one. It's just spray painted. But it looks really great. She did a good job. ?

Brian Heater (03:44)

So tell me a little bit about this upcoming move, why and why now?

Holly Yanco (03:49)

I have been at the University of Massachusetts Lowell for the last 24 years, and it's been a fantastic place to work. And while I've been here, 13 years ago, we started the New England Robotics Validation and Experimentation Center. We call it NERV for short. It's a lot easier to say. The opportunity to move to UMass Amherst, which if you're outside the state, it's like, she's just moving UMass systems. And I am, it's about an hour and a half away. But they asked if I would come out and help them grow their robotics program. And so that was a pretty exciting opportunity to do something new. And actually, what I think I like best about this move is that I'm still going to have the opportunity to talk to UMass Lowell. So the idea is that we will expand the NERV Center to be in Lowell and in Amherst.

So here in Lowell, we've been testing exoskeletons, search and rescue robots, a lot of robot manipulation and a few other things. But what I'm interested in picking up when we go to Amherst, being a land grant school, is thinking about agricultural robots, particularly kind of small family farms. We don't in Massachusetts have kind of the large Midwest farms, that kind of John Deere, but there's a lot of work that the university does with the extension school of family farms. And so can we think about what we could do with robot systems and how we could test them to help small farmers use robots. The other thing I'm thinking about expanding into is ? underwater robotics. We've been doing some underwater robotics here, but ? UMass Amherst, if you know the geography of Massachusetts at all, ? they have a site, they have a Marine Station in Gloucester, Massachusetts, which is actually much closer to Lowell than it is to Amherst. But I want to take advantage of the fact that they have that Marine Station so that we could actually do some underwater work right off of their pier.

Brian Heater (05:28)

What's the through-line through all of these modalities? Like what connects all of these different kinds of robots for you?

Holly Yanco (05:34)

So from my research perspective, I do human-robot interaction. And I always say that for me, HRI is the same as robotics. And then because I have to give it to people, I'll say, well, it's robotics minus epsilon. So you have to give people that little bit. Anything that we do to make a robot better, whether it's better path planning, ? whether it's better tele-op, whether it's better... It can be social. I don't do a lot of social robotics. I always joke that I'm an anti-social roboticist. But I think that all the things that we do to make robots better also improves how it interacts with people. It may not be the primary interaction that we're thinking they'll work with it, but there'll be bystanders to it. So everything that we do to make a robot more understandable, to work better, is HRI. So I think a lot about just how do we build great robots. But then the other part of me thinks a lot about how do I break robots? So this is my mug I'm using right now. This is the NERV Center mug.

So edge cases are really interesting to me, right? So I think, you know, when Google first came out with their automated car, they would say, oh, we drove X hundred thousand miles, whether it was 200,000 and 500,000. And that's great. But if you think that they're doing most of their driving in kind of the Bay Area, that's very different driving than if you're in the mountains of Colorado or if you're in New England, where we don't bother to paint the roads most of the time. ? So how do you actually trust what you're doing for that testing? Maybe it's really, really great in Mountain View, but is it going to be great everywhere else? So how do we think about testing robots? You want to test them quickly, but to hit as many edge cases as you can. So I kind of have this dual side of my world of thinking about how do we build better robots and then how do I break them?

Brian Heater (07:20)

It's a little tongue in cheek, obviously, and you have a mug, but do you feel like to a certain extent it is your role or your job to push robots to the point of failure?

Holly Yanco (07:30)

If we can't find where the failure points are in the lab, we're going to find out where they are once we deploy them. So we're better off to find them before we go for deployment. Whether that's exoskeletons or robot wheelchairs or underwater robots, I think that we need to be testing them in a wide variety of scenarios. But we need to also do it in finite amount of time because we want to also deploy our system.

Brian Heater (07:54)

Yeah, it's funny, you know, HRI, obviously, it's one of those terms. There seem to be a lot of these terms in and around robotics. Humanoids is another good example where everyone kind of has their own specific definition of what it means. But when I think about underwater robots, when I think about agriculture robots, to a certain extent, those things are successful when they're kind of navigating or working away from humans.

Holly Yanco (08:20)

I mean, they are, but look, this is an argument that—not an argument, but a discussion I've had many times with some people at DARPA. When you talk about robots, you don't really want autonomous robots. You want super capable robots, right? We want robots that we can task, that they go off and they do their thing and they come back. So the testing is part of that. But I also think that that also comes into the human-robot interaction. If we have the robots for farming, do we want to tell the robots that, you know, I want you to focus on picking these crops today or checking out these other crops? Because it still needs to somehow make some decision. I mean, yes, we could create the most capable, biggest AI system, but on the other hand, that's more to break at that point. In some sense, it's still going to need to work with a person at some level, whether it's the instructions coming from the person or if it has trouble, if it's having a failure, how does it explain why it's having trouble doing what it's doing? If it gets stuck, how does it call in for help?

Brian Heater (09:22)

That seems to be a big potential for large language models, is increasing natural language interaction between robots and people.

Holly Yanco (09:30)

We've done a little bit of exploration of that space. So in my lab, we've been looking at ways to have robots explain failures that they're making or that they're about to make. So we've used behavior trees and then assumption checkers that came out of Brigham Young University to look at whether if we're already violating assumptions to begin with, then we could say, OK, we're about to fail. So if we're going to do those sorts of things, then we could have templated answers. Templated answers sometimes sound very stilted English and then people say, well, we don't really trust them because it doesn't sound like English. So we started playing with LLMs. So instead of templated language, we tried using LLMs to generate the language. And what we found in our first study is actually people didn't understand the explanations coming out of the LLMs. And I think the issue there was not so much that the LLM wasn't good. ? I think we prompted it incorrectly because we had given it what the robot could do. We had passed the behaviors. We had passed, you know, we basically gave it a developer's knowledge and we asked it to explain to a person who wasn't a developer. So they couldn't understand as well.

So then we tried another experiment where we actually made the prompt saying, explain this to more of a novice. And we had a third condition where we passed in a picture of what the environment around the robot looked like, and then it could ground its explanations. And then people understood them better. So I think there is space in LLMs. I think my concern with LLMs, of course, is how do we know that it's correct? Because if you're going to have the system explain something to a person about a robot, you want it to be explaining it correctly and not making things up. So we need to be pretty careful about that.

Brian Heater (11:07)

Trust is a really interesting word in this context, and it seems like a lot of the efforts to make robots more trustworthy or to allow people to trust robots more are to make them more human-like. So in this case, it's speech, it's natural language conversation. In other cases, and you made a point off the bat to say that you are not really operating in social robots, but is there a way to make systems more trustworthy that isn't necessarily just turning them into a humanoid robot?

Holly Yanco (11:40)

Yeah, I mean, the reason I'm not working on social robots is I feel like there's so much to do just on the capabilities of robots. So if I don't feel like robots are fully where they need to be before we can really have them be out and be social entities. And in terms of trust, I don't think you need to be social to be trusted. One of the things I've been thinking about lately is we've done a lot of work—my lab's been working on trust for the last 10, 15 years—and we've looked at whether people trust the robot system or not and, you know, what happens in failures and what happens to the trust in failures.

But my PhD student, Greg LeMaisery, has done an experiment recently. And what we found is that the robot can fail a lot and the people can still trust it because if they understand that it's going to fail, they can trust it. I mean, if you tell me this robot is going to do task A 100% of the time and task B 0%, I can trust it to do task A and never allow it to do task B. So we're starting to think now about, in addition to trust, what does it mean? How do you measure or quantify capability of the system? Because I think there's going to be a capability component. And how do you tell people what the capability of the robot is? Could we prime users to know what the capabilities of the robots are? Can you do that quickly so when they come up to use a robot, they're pretty well calibrated as to what the robot could do right away?

Brian Heater (13:11)

I'm thinking of like C3PO all of a sudden and him always giving like the odds of everything. That seems like the most, I guess the most obvious way on the face of it is just to tell people, you know, roughly what percentage of the time you succeed or fail in something.

Holly Yanco (13:26)

You can, but most people, let me just say most Americans are not really good with probabilities. Sure. They're not going to understand them, right? I mean, there's a large probability that Americans are going to die of a heart attack, but some are afraid to get on a plane flight because they think that's how they're going to die. But the probability of that is quite low.

Brian Heater (13:41)

I should say nobody trusted C3PO more when he tossed out those probability numbers, so you might be onto something.

Holly Yanco (13:47)

I don't know the probability, but the interesting thing is in Greg's study that he's done recently is we found that people can pretty much identify those probabilities. They're not spitting them out as numbers, but we can see it coming in in the data that when we go from 100% to 70% to 50 to 30 to zero, the bars are right there. People are recognizing how often it works. So I think people are seeing it, but I don't necessarily think they're modeling it as probabilities. So the question is, how do we get people to understand what the robots can do, when they can do it? I'm not sure; for us probabilistic is easy, but for most Americans, I don't know that it is. Outside of the US, they're probably better at math.

Brian Heater (14:26)

Like using the Roomba example, right? That's something that there's an expectation when you spend money on a robotic vacuum that there are certain things that it's not going to be able to do at all. And there are certain things that it's probably not going to be able to do particularly well. But as long as it does the majority of the things that you want it to do a majority of the time, then most people will consider it reliable.

Holly Yanco (14:49)

Yeah, I mean, when I open that bin and there's a lot of stuff in the bin, I feel like it did its thing. Before that, the Scooba, which sadly they don't sell anymore, that water would come out really dirty. It was amazing. It was really good. But it was like, wow, it really cleaned my floor. Look at this water I'm emptying. And the Roombas that I continue to buy are the less smart Roombas. I don't care.

Brian Heater (15:01)

I think that's probably why they don't sell it anymore.

Holly Yanco (15:13)

That's my environment. I'm happy with it bouncing around. That's enough to get my floor clean. But I don't know. It'd be interesting, and I'm sure there have been studies done on whether people... I'm sure they must like the mapping, otherwise why would iRobot have done it, right?

Brian Heater (15:27)

That's interesting in and of itself in that your expectation is just to get the floor clean. And if it's doing so in like a random motion, then that's totally fine as long as it does what it's supposed to do in a given time.

Holly Yanco (15:40)

Yeah, I mean, honestly, most of the time I don't care how long it takes. I set it off and I leave.

Brian Heater (15:43)

Do people necessarily want complexity? Do they over-expect complexity from robots?

Holly Yanco (15:48)

I think the successes that we've had in robots is where we can narrow things down. We can make them work in a very particular situation. Certainly the Roomba was incredibly successful. Tens of millions of robots were sold. That said, Waymo's out there with cars, and I've had fun riding in them in San Francisco and Phoenix, and you can call them. More and more robots are out there, but still I think they are limited, right? I mean, it's very clear when you ride the cars in San Francisco that they've got that model down to pretty much millimeters, right? You can see it when you're watching the screen when they're driving. So it's not going off into uncharted territory to be driving. That's how we deploy robots, where you have to create some sort of limitations.

Brian Heater (16:33)

I think though there is, people perhaps trust something less when there is this—it's almost counterintuitive—but the fact that somebody is sitting in the driver's seat and can take over almost leads people to trust it less because they know that that's somehow implicit, that it's not getting it right all of the time.

Holly Yanco (16:51)

I mean, in the rides I took, there's nobody in the driver's seat. I believe that there was one time the robot was clearly taken over. We sat for about 60 seconds and then we moved forward. So I think it was taken over remotely. I don't know for sure. But I think, I mean, it was actually really interesting because it was a group of roboticists and we were using them and some people were still uncomfortable with it. And my feeling was, look, if I'm not willing to get into a car and let it drive me, why am I working on robots? I mean, I want to trust robots the way my car trusts Totally different to other brands. I want to have trust in the company that's putting it together and believe that they've actually tested it.

Brian Heater (17:29)

That's fair. That's fair. And you do have a history with Google. Again, I was watching this teleop thing that you're doing, and that was like, what, 15 years ago that you were testing out these teleconference robots over at Mountain View.

Holly Yanco (17:42)

Yep. A couple of my students worked there and we tested different ones in Mountain View. ? Gosh, that was a really long time ago. ? That actually led into Kate Sui's PhD thesis where she had taken a Vigo robot and made it a lot more autonomous. That was with the Hawaiian shirt on the robot. What's been really fantastic about my career in robotics is I've had lots of opportunities to deploy robots in lots of different places. My students competed in a competition at Johnson Space Center on the rock yard one year. And that was fantastic. We got to go visit the moon rock facility with people who had been working with the rocks since they had been brought back from the moon. So it's a fantastic career. I always say I have the best toys and then I've got this robot playground to go put them in.

Brian Heater (18:26)

Yeah, I know a lot of people are hesitant to give predictions, but I'm wondering if you and I had spoken about the same subject in 2010 about the prevalence of teleconferencing robots in 2025. Would you expect there to have been a lot more out in the world than there are now?

Holly Yanco (18:43)

COVID was an inflection point, right? Actually, I think in 2010, I would have expected there to be more. And after COVID, what we saw is that it just didn't really find its niche.

Brian Heater (18:53)

Yeah, it wasn't as much of an inflection point as it could have been,

Holly Yanco (18:57)

I mean, in some sense, I think it went the other direction, right? We all went into Zoom. There was no point to have a robot in a place when nobody was in those places. So that was, I think, a strong indicator against telepresence robots, which was kind of unfortunate because it felt like the field was moving along. And with COVID, there just wasn't a use case for them.

Brian Heater (19:19)

Is there a reason? You know, again, it seems like that technology has been around for a really long time. Again, I remember visiting iRobot's office in Bedford a long time ago, and they had some version of that too. Why haven't they really taken off in a meaningful way?

Holly Yanco (19:35)

Well, I mean, I think if you go back 20 years, there wasn't good enough internet in most buildings to support it. So, I mean, maybe 30 years ago. So I think that was one of the problems. You just couldn't get the video to go. I think beyond that, I mean, there's an expense, right? And there's this notion of, there's also—it feels a bit ? uneven. So if I'm in a space and there's a telepresence robot, somebody can come into my space. And that was actually at Willow Garage. They had developed the telepresence robot. And they had a rule that if you were in our space, you had your camera on. So they required people who were driving in their space to have cameras on, because they didn't feel like it was fair that people could come in and see them, but they couldn't see who was driving the robot. So it's kind of this one-sided thing.

I think when you start looking at people who are older, would they want to have a telepresence robot? Sometimes they're a little bit leery about, well, can my kids just come into my space whenever they want? Whereas with a phone, you can answer or not answer. You can, of course, mute a telepresence robot. I think that some of the use cases for telepresence robots where I think it could be really valuable is, Kate Sui, when she was doing her doctorate, was looking at possibilities for allowing people with disabilities to travel to museums through telepresence robots. So if they couldn't get there, they could be there. And she was looking at how you have interactive exhibits that the robots could interact with so somebody could actually be interacting with them remotely. So I think there are good use cases. I know Vigo back in the day was working with letting students go to school using them. So if students were immunocompromised, they could use them. So I think there are cases for them. But I think we got so used to using Zoom and other video conferencing systems that do we need to have a presence somewhere else?

Brian Heater (21:27)

Yeah, and you touched on something again, getting back to that conversation or that talk that you were giving around telepresence that, I mean, it's one of these things that once you say it, it is obvious on the face of it. But when we think of robots, telepresence robots, we tend to think of the experience that the person is having who's remotely operating it, but not so much the people in those meetings who necessarily have to sit with the robots in the meetings. That isn't necessarily as good of an experience as the other way around.

Holly Yanco (21:56)

All right, and I think that the systems that we had tested in Google years ago, neither of them had any mapping capabilities. And you had mentioned the one that iRobot had, Ava, spun out into its own company, Ava Robotics. That one has a lot of mapping. So that does have the capability that once you have that meeting and you're done with that meeting, the robot can go to the next location it needs to go to. Whereas if you're the remote person using it and I have to drive it to the next location or drive it back to the dock when a meeting ends, I can't just get on my email or go to my next meeting. I have to take care of the robot. So there's issues even for the remote person. ? People did feel like they were more present in the robot, that people paid attention to them more in a meeting. Especially, it worked out really well when there was like one person remote and then there was a big group because it kind of brought them into the group. When there were two larger groups, it didn't quite have the same outcome.

Brian Heater (22:51)

So how long were you at Willow Garage for?

Holly Yanco (22:54)

I was not at Willow. I was just talking. I was just giving an example of them doing that work at Willow, but I was not there.

Brian Heater (22:55)

? OK. Yeah, I mean, you've been really on the research side for your entire career.

Holly Yanco (23:05)

Yeah, I graduated from my PhD, spent a year at Boston College, and then moved up here to UMass Lowell and have been here 24 years. Now I'm making a big move to another university. So yeah, unlike my advisor who's done many, many companies, I have not.

Brian Heater (23:24)

Yeah, yeah, who we had on the show is Rod Brooks. He's, you know, stayed in academia for a little while. And I'm sure that there have been opportunities. I'm sure that you've had a lot of students come through with, you know, technologies that could be commercialized. ? Was that ever a move that you were interested in making, even if just maybe dipping a toe in?

Holly Yanco (23:49)

For me, the exciting part about academia is the ability to do lots of different things, ? as opposed to really putting everything down on one thing. And I mean, I've certainly had students go and work in a lot of different companies. So I think stuff in the lab does spin out, at least the ideas do. But there hasn't been something where I've been like, yes, I have to go and start a company with this. I haven't found it yet. Maybe the thing will be there. I really like the freedom to try all the things. Back in grad school, ? I worked at the Digital Equipment Corporation research lab in Cambridge for a few summers. What you need to do to show that your research is of value to the company, that things can be canceled at any time. Yeah, I have to find funding to pay for research, but there are ways—I mean, a student can be a TA and still be doing the research even if you can't find funding for it, right? So you can still do the ideas even if you can't get money behind them, if you think they're good ideas and want to pursue them.

Brian Heater (24:53)

What are those pressures like? I mean, again, I'm kind of coming at this from the more corporate side of things. But as far as you or a student having a good idea, is it something where you necessarily need to reach out to the people giving the grants or giving the investments to get past them? Or are you really able to just sort of work on a project and see it to fruition based on whatever grants you have?

Holly Yanco (25:17)

Yeah, I think it depends on where you stand in the funding cycles. I mean, certainly, we'll write proposals for things that we've got ideas for. I'll bring in students for things I have ideas for. There's always—I had another student, Mark Massiri, and he was super interested in doing multi-touch to control robots. And we could not find a funder who wanted to pay for it. ? So we found other ways to cover his time. And ? he did a great thesis. He went off and worked for NASA and DARPA for a while and now is in Australia. ? So I think that, you know, there are ways. I think the thing that we become really good at as professors is telling a story about how everything in our lab fits together, even though the students don't fit together at all. But I mean, I think the funding may look different moving forward. We'll see. Hopefully, US funding for robots and AI— I think that there's silly indications that the administration is supportive of AI, which is good.

Brian Heater (26:18)

Yeah, is there, I mean, is there generally more wiggle room if it is like a government grant versus a corporation coming in?

Holly Yanco (26:25)

It really depends. Some corporations will give unrestricted gifts, and then that's a wiggle room as well. It's really finding a way to enable—I really look at my role as enabling my students to do the things they want to do. I need the resources for them to help them out.

Brian Heater (27:13)

Every time I'm writing something up about a research project and then, you know, finally come to the end and then you start talking about like, oh, these are the applications that it could have in the future. You hook onto it first because it looks like a cool technology. And then you get to the actual like efficacy and the usefulness of the product at the end. Is that kind of real world usage, whether it's commercialization or anything else, something that's at play from the beginning of the project?

Holly Yanco (27:13)

It should be. If you're designing things that are going to work with people, you'd certainly be talking to your intended user groups from the beginning and then working with them. So we've certainly done a lot of work with participatory design and trying to get feedback from the intended groups. Of course, we all in academia fall back on our convenience populations of testing things with undergraduates because that's what we have. ? But really, if you want to do good work and think about what could be deployed, you need to be working with the intended user groups. I think that one difficulty in my world—I do a lot of work in assistive technology—and it's just quite difficult in the US to get insurance to pay for a lot of these systems. ? So, you know, people have been working on robotic wheelchairs. I did it for my thesis, you know, whatever it was, a long time ago, over 25 years ago, and people have been doing it since, but there's no real push to get this out there, right? So if you need assistance with driving in your power chair, then you probably need assistance and you need a person to help you. And then they just say, well, why do you need a powered wheelchair if you are paying for someone to help you? Sometimes it's hard to find the business model for some of these technologies as well.

Brian Heater (28:27)

Yeah, yeah, I feel like every couple of years I see like a cool new robotic wheelchair and, you know, it doesn't necessarily go anywhere. ? It seems like exoskeletons though, like at least to a certain extent, certainly in commercial settings are finding their place now.

Holly Yanco (28:45)

We've been doing a lot of quick looks for the Army on different types of systems. So there certainly are a lot of systems being developed. ? So we do a lot of, you know, we bring in just, you know, six or eight subjects to try them out in different cases. Because of course, all the companies say, we can do X, Y, and Z, but we're trying to do a consistent evaluation across all of them so the Army can be taking a look at them. I think that even then, you know, there's certainly—the military has been working on trying to get exoskeletons now for what, decades, right? Yeah. I think that, you know, maybe one of the differences now is that they're much more focused on one or two joints. You don't see sort of the whole-body, these Sarcos big exoskeletons that you saw 25, 30 years ago. And even then, you know, there's a lot of work trying to do predictive modeling about what the person is doing so you can provide the right assistance. But that's hard, right? If you think of having an exoskeleton where you're running and then you stop, even the best prediction in the world, there's just a lag, right? So you might have to make sure you relax your knees so you don't get that last push after you stop. These are really hard problems to solve.

Brian Heater (30:01)

My father was diagnosed with Parkinson's last year and I was doing some research on some of the exoskeletons around Parkinson's and then that presents its own challenge, in that there's a way that people move forward and there's kind of a special sort of predictive algorithm that you need in order to continue that motion going forward.

Holly Yanco (30:19)

We don't think anything about how we're moving and we stop and we turn. But the system has to get that through our muscle motion, right? And it has to have that second or two to ? get the sensor readings and process it. So it is a really hard problem. I think that people are making some strides in that, but there's a lot to still be done there. I mean, I think it's—I think assistive robots is a really interesting thing to be working on. I think that from an HRI perspective, some robots are far away from you. Some of them you ride on, and then some of them you're really integrated with them, like you are with the exoskeletons. That creates a very different sort of human-robot interaction.

Brian Heater (31:01)

As you're making this move to Amherst, will there be—I guess you kind of have to just start over with a new group of students, is that right? Or do you get to take some folks with you?

Holly Yanco (31:12)

So I've been chair of the minor School of Computer and Information Sciences here. So I actually didn't pick up new students for the last three years, so I'm finishing up my students here. So it really is kind of a fresh start over there. I'll be finishing my students; they'll be staying here. But one of the things I'm going there to do is to help them grow their program. They've got several faculty doing the work in robotics, but they want me to come in and help expand, bring the NERV Center there to think about how we could have more academic programs in robotics.

And I think it's interesting to look at bringing robotics to the Pioneer Valley. So in Massachusetts, you've got Boston and Cambridge, and then you get 128 around it. And that's mostly where most people want to stay. Then you have 495, and Lowell's out on 495. And then you go farther west, and then you're in Amherst. More people want to be near Boston, but I think that there are a lot of people that, you know, we could bring companies out west and there's certainly people working on that. And I think it would be interesting to see how we can expand robotics out there in the Five College area.

Brian Heater (32:20)

Obviously the conversation around brain drain has changed quite a bit post-pandemic, especially with people increasingly working remotely. What is your pitch to get people to sort of move outside of the greater Boston area?

Holly Yanco (32:32)

So I'm getting an apartment in Amherst. So I actually am not the one that's in greater Boston, at least not the first year. I think there's a lot of talent. I think that, you know, real estate's cheaper, right, if you want to have a company out there. I think that there are five great colleges in the area out there. I think that's an opportunity for growth. I think that looking—you know, Massachusetts is doing a lot of investment in AI and robotics, and the Commonwealth is interested in investing outside of Boston as well, right? They certainly invest in Boston, but they're looking to have all of the areas in Massachusetts benefit economically. So I think that there are a lot of people who'd be interested in working in robot companies out there. ? Good quality of life, bike trails, hiking, cheaper. ? I'm not sure the real estate is that much cheaper at the university from what I've looked at. ? But talk to me in a year and I'll give you a better pitch on Amherst once I've been out there for a little bit longer.

Brian Heater (33:36)

It's going to be an interesting ramp up period too, right? I mean, if there isn't really as much of an existing robotics program, it's going to take a few years really, I guess, to get to a critical mass for you.

Holly Yanco (33:50)

Yeah, I mean, there are six core strong faculty and then many others working in robotics. When I started at Lowell, I was the only person doing work in robotics. So, you know, it took a little bit of time. I think the biggest thing for me is they're renovating a lab for me. Don't get me wrong. It's nice to have a renovated lab, but it won't be ready for six to 12 months. And so I guess it's put me back to the beginning of like, holy smokes, I'm going to have a new lab. But maybe that's exciting because it gives you a chance to rethink some things. What are the things you want to keep and what are the things you want to change and what do you want to be focusing on? I'm really excited about working with my new colleagues out there. They also have obviously very strong groups in machine learning. The Turing Award just went to Andrew Barto for reinforcement learning and many people doing AI and computer vision. So I think there's a strong group of researchers to work with out there. And then they have nursing and physical therapy and agriculture, which we don't have here at Lowell. I think it's… And I'm not abandoning my colleagues here. I'm planning to work with the people I've been working with here, right? So it's a way to expand what I'm doing. ?

But I guess I want a challenge; I want to grow some new stuff, ? creating an undergrad program. One of the things I've been thinking a lot about is here in Massachusetts now community college is free for everybody. And how do we create a robotics program at the undergrad level that takes somebody who went to community college for two years, maybe in a robotics technician program that doesn't have a lot of math? And how do we think about turning that into a four-year degree? It's not simple, but I think it's a huge problem to look at.

Brian Heater (35:43)

Yeah, well, and that's really the start of a larger, really important conversation around upskilling, right? Around as more robots are entering the workforce, trying to find some of these ways to get people who don't necessarily have a background in programming or math or robotics to learn how to work alongside these systems.

Holly Yanco (36:02)

But I think even working alongside them, is there a way to have people… Is there a better way to program the robot so that people who don't have programming can… Certainly there are teach pendants and those sorts of things. But are there other ways? Are there ways that people could give language instructions to the robots, that they could kind of prompt an LLM in some sense, right? ? But of course, you'd have to make sure that things work.

Brian Heater (36:30)

Yeah, I mean, you must have learned a lot on that front when you were doing the robotics program, right, as far as how to appeal to serve a larger group of people and people who don't necessarily have that robotics background.

Holly Yanco (36:42)

Yeah, I mean, I think actually Artbotics goes back—that program, we were working with UMass Amherst. I have these ties from a long time ago. But I think that one of the things that we thought a lot about in Artbotics was that students come to college like, I'm going to be in computer science, I'm going to do video games and robots. And we set them down in Computing I and make them do compound interest tables. And so instead in Artbotics, to teach people for loops, we had a small round chassis. And we had students basically program a pattern and then loop it so you can create Spirograph-like patterns. So there was a reason to do loops because you wanted to see something happening multiple times as opposed to compound interest which, especially when interest rates were really low, was not compelling.

So I think that we—Artbotics also really focused on kind of sensors and thinking about how you wanted it to interact with people and the idea that also sensors help you not to have the motors wear out because they're not running all the time. So I think that there are ways to go towards that. I think that we as a robotics field, certainly with probabilistic robotics, have moved super math-heavy, which is important. I mean, these are important things that we're doing, but are there other areas of it that we could open up? I've long wanted to think about how we could have, you know, robotics classes for MBAs. Those are the people who are going to be buying the robot systems. What do they need to know about them? What do they need to understand about them? What questions should they be asking?

Brian Heater (38:15)

Yeah, or the people becoming like the CTOs of robotics companies as well in a lot of cases, like executives coming over. What is that—I mean, there's obviously there is some math background for a lot of MBAs, but how do you bridge that gap?

Holly Yanco (38:29)

Yeah, I mean, look, I don't think everybody needs to be programming them. Not everybody needs to be doing linear algebra, but I think that even just knowing what questions to ask and to think about—maybe I'll teach a whole bunch of MBAs about how to, what can you do to make this robot fail during the demo? ? Maybe that would be a great outcome, because you've got a company that can break things. You can't just accept—and I think this kind of goes back to what the Army is doing with the quick looks—you can't accept what companies are telling you, right? You can't, you know… Or if you look at, like, small unmanned aerial systems, then they might say, it flies for 15 minutes, but it might fly 15 minutes when it's not running any of its sensors, right? So you need to be able to test those against one another and kind of have a standard way of looking across all of these claims. So maybe that's the thing for MBAs. So these are things I'm going to start thinking about. I'm going to visit the University of Michigan in September because they've created a good undergrad program. So I'm going to go visit them and look at that and think about how we're going to do that and what does it look like to create a robotics master's program when visas are tough right now. So what does that look like? Will it change, hopefully?

Brian Heater (39:48)

Yeah.

Holly Yanco (39:55)

Obviously we need to bring more talent into the US. But, you know, even—how do we focus on community colleges? And then that's focusing on more US students too, as well. So I think everybody needs to be in robotics, of course, and finding the pathways for everybody to get there.

Brian Heater (40:12)

I always hear these great stories from folks who are in ag-tech robotics about really sort of like getting out—like literally getting out into the field—and meeting farmers and, you know, just sort of the new dimensions that you learn about the work that you're doing. Is that, as you're taking a look into agriculture, part of your own plan for yourself is to kind of get out of the laboratory a little bit more and actually interact with these more in the field?

Holly Yanco (40:41)

Yeah, I think that that goes back to what we were talking about is, you know, talking to the intended user population, right? If you're going to be developing systems, then you need to find out what the problem spaces are. And certainly, you know, Joe Jones, one of the creators of the Roomba, he had done Harvest Automation and gone out and talked to people about moving pots in the commercial growers and stuff like that. So I think, you know, it takes a lot of effort to go do that. Maybe that's a really nice crossover with those business students. Yeah, because they're starting to identify what those problems are and those issues that need to be solved.

Brian Heater (41:18)

Yes, so we are starting to come up on time, but people will always ask you and you'll always ask yourself during different points of your career, you know, if I had to do this all over again, what would I do? It feels like you're affording yourself—or the schools are affording you—the opportunity to do that, to really in some ways start from scratch with, you know, decades of knowledge. What are some of the things that, in starting the second NERV Center, you're not going to do over again?

Holly Yanco (41:48)

We're not going to count on one company using it so much that it pays for everything. That did not work out. And we pivoted pretty quickly and moved more towards grants and finding more customers. ? At the time we created it, it was one person, one company that said, we'll be in there four days a week. They were not. And part of that was just because it took so long to start it up. By the time you get the space and you build it up… So I think that that's one thing: thinking more of NERV supporting people who are developing robots, but hopefully—you know, we've worked a lot with NIST and I hope NIST will still exist and be able to do this. They've, ? there's not large money there, but they've really been investing in thinking about how we test robot systems and how we validate AI and robotics.

You know, one thing that I don't have to do all over again is because I'll still be working with the NERV Center here at UMass Lowell is that, you know, I still have a strong group here that I'll be working with as we expand to have more people out there. So it is starting from scratch and it's not starting from scratch. ? I think the craziest thing is my ? iRobot ATRV Jr., which was the robot I bought with my startup package here 24 years ago, is moving with me to Amherst because I'm still going to be using it. That robot is really solid and it has a full-size computer chassis inside it. So we just keep updating it and that platform is… People are like, you're still using the same robot after 25 years? Yeah, it's a good robot. But you can't make money off of academics in robotics, right? Things keep going for us.

Brian Heater (43:24)

I know you're using Fetch. Obviously Fetch isn't really a going concern as much anymore. ? The Baxter robots were big in the academic world. ? What is it, Unitree? I know there's probably actually some issues now with some of these, like the back doors and maybe working along some Chinese companies.

Holly Yanco (43:44)

Yeah, we have Spots. OK, we didn't go with Unitree. So we have a couple of Spots and I'll be getting those as well at UMass Amherst. Honestly, hands down best robot I've ever gotten out of the box. ? It just works. ? So we'll be doing that. The UR5, I mean, certainly there's a lot of—you know, where we are now with ?

Brian Heater (44:06)

That's an old, old robot.

Holly Yanco (44:09)

I mean, it's manipulation, right? It's a good robot. It's solid. The Universal Robots are solid robots, but there's a lot of different manipulators to use. We have a number of them now at NERV in what we call the Armada, which is a little bit of a joke because we started—we were working with the Advanced Robotics for Manufacturing Institute, so we called it the Armada. We wanted ArmFarm, but Google got there first. Yeah. So, you know, lots of different robots and some old, some new.

Brian Heater (44:31)

I'm so sorry.

Holly Yanco (44:39)

But I think we're using Stretches. I'll be using those in my classes and the Spots. I don't know beyond that. I don't know if there's a platform that I'm like, wow, I really need to get this new platform right now. I think a lot of the robot companies are looking at actually developing them to put right into an application, and much less so for us academics. And some companies don't want to sell to academics just because they don't want us publishing something that says something bad about their robot, which I get it. I do. But if you look at now versus, you know, whatever, 30-plus years in grad school, when people had to build our own frame grabbers… I mean, and now you can buy all sorts of different robots and, you know, tons of simulations out there and tons of companies working on it. And, I mean, just ROS and what that enabled. I think we're—I think we're in a great place. I ride in robots when I travel. It's pretty—the robot cleans my house and I ride in robots when I travel. So I think it's, you know, I'm bullish on robots. I mean, it'd be kind of weird if I weren't, ? but I think there's still more to come. It's going to be great.

Brian Heater (45:58)

And I have to ask you—every time I hear you talking about Armada, every time I hear a really good acronym I wonder how much of that acronym was sort of back-renamed and fudged—and NERV is a really good one. Did you have to work your way backwards into NERV?

Holly Yanco (46:11)

So we actually wanted to do NERD originally. So we wanted to be the New England Robotics Development Center. But at the time, Microsoft—I don't know if they still do—but they had the Microsoft NERD Center in Cambridge. And so we're like, we probably shouldn't overlap with that. And I was saying this at a meeting, and someone who worked at BU at the time said, well, what about like NERV, New England Robotics Validation Center? And it's like, well, that's like experimentation. So it kind of came by talking to somebody else. So I wouldn't say it's a backronym. It was certainly helped along. But I love it. And people are like, you're nervy. We do occasionally get calls about people with nerve problems. And I feel really bad that we use the name NERV. ? But no, it kind of went from NERD to NERV without the "and," and NERV with the "e."

Brian Heater (47:01)

Professor Yanco, thank you so much for joining us.

Holly Yanco (47:04)

Thank you very much. Enjoyed talking to you.

Brian Heater (47:06)

Thank you so much, Professor Yanco. Thanks to you for tuning in. If you like the show, please like and subscribe and/or rate and review it on the platform of your choice or platforms of your choice. We also publish a sister newsletter every Thursday. You can find more information about that and the podcast over at automated.fm. Thanks again and we will catch you next week for another episode of Automated.