Designing the Machines of Disney’s Avatar with Ben Procter

Transcript

Ben Procter (00:00)

Part of what we had to think about is the fact that Mr. Krabby Pants has ? more legs than a person does, right? In the Amp Suit, the presumption is that there's something that might be analogous to two legs controlling two legs, which of course doesn't work when you've got four legs on the vehicle. So we came up with the idea that it's kind of like you're riding a creature, right? So imagine that you're riding a horse and you have big robot arms kind of strapped in front of that horse. So your legs are going to be telling the creature where to go, along with some control sticks and stuff that are built into the end effectors but your arms are directly controlled as opposed to the indirect control. So we built different paddles and levers and stuff into the leg area of the seat so that you actually use like the same way that you might use your knee as to put pressure inside of a horse to make it move away from pressure. That's exactly how you do it gradually.

Brian Heater (00:46)

Hey everyone, my name is Brian Heater. I am the managing editor at A3. I hope you're having a happy whatever it is that you do or don't celebrate.

This episode of Automated is very special for a couple of reasons. ? First and less interestingly, it was recorded at the Ramada in Sunnyvale, California during a late checkout. The guest was actually booked the night before while I was moderating a panel about AI over in Mountain View. The second reason is, of course, the guest himself. This week we are speaking with Ben Procter, an art director production designer and visual effects artist who has worked on some of Hollywood's biggest blockbusters from the Transformer films to Prometheus. We're here to talk about another absolutely gigantic franchise. This one actually has a new entry due out later this week, Avatar: Fire and Ash. I note during the interview that this is a conversation that I have actually been thinking about since I saw the last Avatar, the second Avatar in theaters. It's a great conversation. I hope you enjoy it as much as I did. Don't forget to like and subscribe and we will see you on the other side.

Brian Heater

So as I mentioned to you on my initial pitch, I saw the last movie in theaters. I was sitting there with my 3D glasses on. ? I should mention this was a Manhattan theater, so, you know, probably a $50 ticket. Enjoying the movie. I think they had just landed on Pandora and I see these little robots come out, little like spider or crab-like robots come out and they're doing building and it occurs to me at that moment that the next time an Avatar movie comes out, we absolutely have to get some of the behind the scenes folks on to talk about them. Because the first thing that jumped out at me is, these are very much based in robots that we have right now, like robots that are out in the world and research that's being done right now. So those are the hex bots. When you start production or start the creation on a robot like that, how much are you actually looking to the real world and current robots for inspiration?

Ben Procter (03:17)

Yeah, I mean, the idea that Jim wanted some kind of swarm assemblers that were moving around, I believe existed previous to us doing design on them. don't think, like there are times then we just come up with an idea and we pitch it to him and he's like, great. And then it goes in the movie and that's awesome. I think this was an idea he had already. The idea that they were insectoid or insectile, whichever is the correct word, in some form was already there. Jim is very familiar with technology and we're, you know. Those of us who work on hard surface designs for movies are also big nerds and geeks for these kinds of things. you know, the idea that didn't have to be humanoid in any way, right, that you can have a swarm that operates together in a kind of, you know, algorithmically driven, intelligent way that's grander than any intellect that's on board of each of them is an idea that we're all familiar with. And so we kind of went with that. We came up with a family of different designs, figuring there would be need for different weight classes, essentially, like things that could handle and carry larger pieces of material up to the job site, which could be 30 stories in the air so that the little friends could come in with their welding torches and their fastener drills and whatnot, and attach things into position.

At the end of the day in the movie, Jim wasn't interested in showing that there was like a group of different types. He just didn't care. you know, the idea, he's like, I think it's more of like a von Neumann machine kind of thing. you know, as if deliberately, every part of the process is, you know, they're identical little units, right? ? But, you know, I enjoyed all the exploration we did. We tried to break, you know, the kind of typical expectation. You you watch WALL-E, right? And they're kind of cousins of WALL-E in a way. Of course, they have six legs and they operate very differently. But as far as a yellow construction related robot of a certain size, ? you know, they have things to relate to in media and pop culture. But of course, we want to move way away from the WALL-E thing of trying to personify these things. We want it to antianthropomorphize these guys. So they don't have a face of any kind. We wanted to keep it a lot of like little cameras and sensors, a lot of little tools. They don't even have a front or a rear end that is dominant. There's probably, I think whichever three quarter angle you see it at first, you could easily perceive that as being the front, but the other end is just as equally studded with sensors and different types of tools. So I think it's in the movie that you could see it, you know, one of these guys doing, doing a job and then spinning around in position so that it's what was formerly its butt end becomes the now operating end. And so anyway, we just leaned into all the potential of these kind of freeform little dudes. And then there was a great sort of texture to have on our buildings in the aerial shots. It takes a lot of them because they're so small. It takes a lot of them to be visually noticeable in these wide aerials, but we got them in there. So they're crawling everywhere.

Brian Heater (05:53)

That's a really interesting balance that you have to strike because obviously, by and large, the robots in this movie are like many of the robots now are biologically influenced, right? So they do they are insectoid, as you said. So you have to at once both make them biologically influenced, but also make them like as mechanically presenting as possible.

Ben Procter (06:14)

Totally. Yeah. mean, everything we do on the tech side or the hard surface side on Avatar, you know, as anybody can tell, it's always like detuned from the most sci-fi, most futuristic version that could be, right? You know, obviously we all have computers in our pockets that look like slabs of abstract glass, right? This is where technology is really headed. But as a kid who grew up watching 1980s movies, some of them being Jim Cameron's; I became familiar with his aesthetic from the power loader and other things like that, leaning into kind of hardcore industrial stuff, A, makes things feel much more real. B is sort of timeless because there's, there's a certain look to very, very functional things, whether it's stuff in a space program or stuff on oil rig or stuff that's used for manufacturing. Anything that's not consumer facing has a kind of a vibe to it. That's that for people like me is really cool. And if you can lean into that ? then you won't fall into kind of sci-fi tropes of trying to be futuristic too hard, right? And then we did things that are geeky, like adding carbon fiber patterning on certain elements of our designs. The scale suit is meant to be completely carbon fiber in terms of its cowlings. And you can see that very subtly in certain places where the military paint job doesn't cover it up. We're nerds and we lean into the sort of like...whatever, poor man's ignorant version of being really aware of how engineering works. We do our best as kind of fanboys or fangirls of this kind of thing. I'm sure we get a lot of it wrong, but at the same time, I'm sure we're inspiring people who really work in the field as well with the things that we do. So it's a good balance.

Brian Heater (07:53)

So you're not calling up MIT to have them consult on the specifics and the physics of a robot.

Ben Procter (08:01)

We don't… I would say we typically call up experts ? in certain categories. It tends to be medical graphics, to be honest, where we call up experts. So when we needed to show how the link system would work with coordinating two different brains to come into synchronicity, right? In the first avatar, we consulted with neurologists then, like, what would the activity patterns look like? What is the correlation? And what's the jargon we could put into our motion graphics of what these...parameters are that are coming closer together and finally into a full congruency link. And so we did it again with, you know, in Avatar 2, you saw a giant brain imaging system when Spider's getting, you know, he's on the rack, you know, they're probing his memories. We needed to know if someone's trying to probe your visual cortex, what would be the networks that would be active as you fight that? And I'm getting totally off topic for your podcast, but you know, we do go the extra mile in terms of research on things that I think we are least familiar with. I try to be a scientifically aware, but I'm no neurologist. So that stuff is very helpful.

But for the mechanical stuff, I think it's a lot of research. know, there's so many incredible YouTube videos out there. You know, all these companies that make robotics are doing a great job of promoting themselves, whether it's I don't even know if I should name names, but anyway, the big the big companies that do great videos and experiments. And so, we watch these things and. Yeah, think we, mean, most of us will have done that for fun anyway. So, so we didn't even have to lift a finger to, to, be relatively prepared. And I think it's also just, just like my co-designer Dylan, who handles all the organic stuff, the beautiful flora and fauna and all that, you know, he's a person who spent his whole life, like when he's in a natural environment studying, like the shape of a hillside and the bark of a tree and all these kinds of things that a geek like me, you know, or a geek for different things.

Maybe it just doesn't process the same way, but I've spent my whole life stopping at construction sites and staring at the equipment, looking at how the treads are linked together, ? getting off on the patina of how mud gets spattered onto it. mean, this is weird shit, but this is literally what qualifies you to do the job. ? And so you're leaning into a lot of your background, knowledge, and passion. So these little bots needed to be insectile or insectoid but they needed to look like construction equipment. And I know what that looks like. So we had to keep some of the forms a bit boxier in addition to the more insectoid kind of curvy shapes. You're always trying to find some kind of form language balance so that the eye sees both things at the same time, right? You don't want the eye to fall down a black hole. like, I entirely understand that. Is that even a bug? Like, is that a big bug? Where am I looking at? Or, that's boring. And that looks like something I saw on my street yesterday. You've got to find that really cool balance.

Brian Heater (10:39)

Something that bears mentioning is that the vast majority, if not every single piece of technology that we've seen in the film so far is a military piece of technology. And obviously, one of, again, if not the key through line through all of these films is this conflict between military, ? but also this urbanization, this desire to build in the natural world. that's a big part of the reason why it is really important to not personify these ropes.

Ben Procter (11:10)

Totally. Extraction is a word that comes to mind also. It's like a lot of the stuff we do, whether it's a horrifying drill that we designed that goes up into the poor Tolkien's brain to take out the Amrita and Avatar too. we look at everything on Avatar, whether it's a fish or whether it's the way that we build a Na'vi hut, which we of course we call a Marui. We have a Na'vi word for it. Everything has to be designed as if it's real. The thing that rings in our minds as we get into discussions with Jim is like, okay, what would this really be? You know, it's not kind of like, what's the coolest thing we can think of? It's like, what would this really be? So the Marui has to have a whole bunch of fabric lashed up over the roof that we're never going to see unfurled, but it's for hurricane season. We put those kinds of designs into it because Jim insists that you think through the life cycle, whether it's a creature that has to feed a certain way, it's a machine that needs to be refueled, maintained, rearmed, right? Like we put the details in that allow you to see that the thing has a life beyond the film, beyond the screen. If each of us in our different creative departments, including costumes, including everything, we all do that job really well. It just gives these movies like a verisimilitude that I think is maybe not consciously available to the audience, but certainly subconsciously underpins the story as being like real. Like you feel like you've gone on an actual journey to an actual place. Pandora's real, therefore I care about it more. Therefore it's the threat to it is that much more sinister.

And for me doing the human designs. Which by the way isn't all just about death machines I do a lot of scientific related stuff with motion graphics and things where I actually try to like like use High-tech to sell the beauty of the planet to the beauty of the biology and all that which is a fun other kind of you know sort of lens on the the tech side of things but You know in my in my world. I'm trying to make it all feel real So that in some sense it's relatable like it's very easy to her person to watch a Star Wars, right? I know not I'm not shitting on Star Wars by any means brilliant design, you know, world ever.

Brian Heater (13:06)

Star Wars doesn't need your support. They're doing fine.

Ben Procter (13:09)

They're doing okay. Yeah. But you can't think like, that could be me. Like looking at a stormtrooper that flies in a giant triangular ship and whatever. It's not immediately relatable. My hope as the guy that does the human stuff ? is to make it almost like, oof, like you can almost see us. You can see yourself, you know, in what the humans are and what they're doing. ? You know, if anyway. It's very much a reflection of what we're doing every day on Earth, right? I mean, that's just a fact. So the more I keep it looking kind of like that, it reminds you that, a minute, don't be so proud of yourself that you side with Jake Sully and the Rebels because the minute you leave that theater, you're contributing, you actually work for the RDA, most likely in some metaphorical form or another. You you're contributing to a society that is doing all these exploitative things right now. And we don't try to jam it down the viewer's throat, but that's the backstory.

Brian Heater (14:01)

I suspect sort of another lie that you have to walk again as somebody who is like, as you said, a deep nerd or a deep geek about a lot of stuff is like how much time you want to spend on a specific project. know, between the first, infamously, there was quite a bit of time between the first and the second, but between the second and third, you know, it was just three years. You know, there's all this lore built up around some of the systems that you sent to me. Obviously there's like the video game, there's the theme park, so you do have to accommodate for other aspects of this world. But there is a possibility that you could potentially spend too much time developing and thinking about any given thing.

Ben Procter (14:41)

For sure. mean, look, it all has to serve the story. I, you know, it's with disappointment that I found that the, you know, the story function that the human tech serves in these films, you know, has become maybe a little, a little more constrained over time. You know, it doesn't take a lot of screen time, let's just say, to convey that the RDA is back kicking ass and doing terrible stuff. And so, yeah, there's points at which maybe we've gone a little far given the screen time, but you don't, A, you don't know that in advance, right? You don't know what the final cut's going to be. I mean, there's giant things that get cut out of these movies that they're like, heartbreakingly deeply imagined and even things that got photographed and whatever, they get cut out. you you're still trying to design for the world, but of course, keeping the story in mind. mean, you know, I'm focusing on the, you know, the story aspect of things like this scale is an important design, you know, not just because, wouldn't it be cool if the troopers had like these exoskeletons as a way to upgrade them? That's cool, right? But the real reason they're there is because you need someone who can relate to Na'vi. You need basically Na'vi sized entities that will fight Jake Sully on the tilting deck of the Sea Dragon in Avatar 2. You need these characters that can wrangle and capture Na'vi, right? So it's a very, very pragmatic decision for the story, you know, just to give one example.

Brian Heater (15:53)

Something that I was thinking about as well because obviously to a certain extent up to this point we've been talking about these autonomous systems but you also alluded to obviously aliens very famously you know we all have this image of Ripley in the mech suit and that's carried on over here. Mechs you know these human pilot suits play like a really big part in here and from a storytelling perspective it makes a lot of sense to have a mechanical suit where where a human being is present.

Ben Procter (16:22)

Totally. Humans are puny compared to Na'vi, right? So certainly the amp suit needed to be something that, mean, Jake Sully's like nine feet tall or something around there. We ended up doing a slight rescale of the characters for the sequel. So he's a little shorter than he used to be, but he's around there. you know, obviously the amp suit with Porridge and it was meant to be an overwhelming opponent, right? Something that it takes basically Nate Terry and Jake and that poor Thanator that ends up dying. You know, it takes all of them to take on this thing. ? But we needed a more kind of apples to apples, you know, even fight element for the sequels. And that's where Jim came up with the idea of the scale. But obviously there's a lot of like, to get geeky, there's a lot of specifics to the way that a human drives these different vehicles, right? That they actually interface for the driver. ? which, and that ends up important for the story because you're going to see it like literally, like when Corch is fighting Jake and holding a knife and whatever you cut to the inside of the cockpit and there he is pantomiming essentially, but in the haptic interface that's in that glove and theoretical way it works in the movie, he can feel the grip of the knife in that hand. so giving the actor an opportunity to act and emote and perform the actions of this robot and to have those things correlate in some way that reads well for the audience. ? and yet still work within the constraints like. I don't think there's enough room in that cockpit for him to throw a really long punch as an example. I just don't think you could, you'd hit the glass, right? So there is a non one-to-one kind of ratio gearing, so to speak, of how he's controlling the robot. And we ran into the same things on the design of the scale suit. The kinematics of the scale suit were a big pain to figure out. And we started off with different versions of it and kind of evolved to where we are because I'm like, well, I guess maybe with both, you know, in a wide shot. even as exposed as the operator is in the scale suit, the eye doesn't pick up on the human and then move on to the scale. The eye picks up, this is just evolutionary biology, whatever the hell, picks up on the biggest silhouette and sees that as the body, right? So essentially, in terms of how we rig it and the way that it's rendered in final setup, we of course capture, we capture people and then map that motion onto the scale, right? So that the scale's behavior is closest to the performance of the actor. And then the little human body that we see visually is actually in some sense going along for the ride. It's being puppeteered from the outside in, but figuring out the ratios and angles and stuff and the connection points of the operator to the scale to make that look good to where if I hold up my arm, you know, like this, the scale throws up. It's totally differently proportioned arm with a much longer forearm and all this kind of stuff. It's it's you need those angular relationships to look the same like in an instant read. And then it looks great. But when those angles were different, it it looked terrible. Right. we ended up, know, Jim wanted to do something a bit more like the power loader where the hands would grip something that was attached to the forearm of the scale. And it's what ended up happening is when the scale did any, it was like putting suspenders on the scales arms, right? When it put its arms up to do anything at all, the person ended up like, as if they were on an exercise machine in this crazy wide pose. It looked terrible. Right. So we had to go back to what worked for the amp suit and put a Waldo system onto the scale drivers. So they actually are wearing a very lightweight articulated Waldo system with haptic gloves that are a little slimmer and more sophisticated than what's in the amp suit, but not dissimilar in terms of how they function. And the actors weren't wearing very much. So what they were wearing for photography is basically that apparatus because it's so closely tied to their body. It doesn't constrain them, but I think it's good for them to be aware of the collisions and all that kind of thing. And maybe some shadow casting elements that we kind of have to attach to them off of a back rig so that like there's a If it's a sunlit scene, like when Ard Moore's walking in her scale in Avatar 2, it's a morning construction set kind of a vibe, right? So it's diagonal sun coming. We needed to have something that was moving with her that would block that sun. But most of the time it's bare bones. It's stripped down to just this apparatus, you know, with kind of a harness and a wall in.

Brian Heater (20:20)

It sounds like you're saying that there are examples where, you know, it actually gets to the point where you're in production, you're doing shooting, you're looking at these actors doing the scenes, the actor looks silly, so you actually have to go back to the drawing board and redesign the...

Ben Procter (20:33)

You try to predict that stuff in advance. You're basically doing tests. You're doing mock-ups with foam core and you're doing stuff with the cheapest possible methodology to discover these problems upfront. The Crab Suit has its own Waldo system, which is a weird one that's kind of inside out in the sense that it's in front of the operator instead of behind. I'm giving you some cool photos of how neat that thing looks and how the hand system articulates with kind of joystick, but then a three finger articulators because it's a three finger crab, right? So you don't need all five fingers represented in a glove system. We mocked that up in plywood to begin with. And there's all kinds of things you discover, like the degrees of freedom that you thought would work as it turns out, like it binds, you know, and it still kind of does a little bit in the final one, but we were able to re-engineer the pivots and the lengths of elements ? using the human body sitting on a piece of plywood that, you know, it's held together with, with, you know, like really simple hardware ? so that we could make sure the person could do. Let's just say we have a scene where there's a Na'vi on top of the back of the crab suit. There is one in Avatar 3 coming and the guy is trying to get this character off the back of it, right? Well, you know, the scene is about not being able to do it. So the fact that the Waldo constrains the motion actually suits the story, but at least the actor is able to do something that feels appropriate to kind of this crab thing trying to swat at its back carapace area, you know?

Brian Heater (21:53)

As much as possible, you're trying to develop these systems and finalize them before the actual acting happens.

Ben Procter (22:01)

Yeah, we have to it's too late to fix stuff on the day. If you fix fix stuff on the day, you're burning daylight and you're you're you're in big trouble. Daylight.

Brian Heater (22:10)

There's the issue of burning data, but I'm also curious, like from an actor's perspective, like how do you work with them and how closely do they actually have to study your designs to know how to act within a mech suit?

Ben Procter (22:22)

Well, that's a great question. And it leads into the fact that we have a pilot's manual for the Crab Suit because that was a less intuitive setup, I think, than the AMP and the Scal, which are much more humanoid in the way that you're relating to the Mac. ? But I'll go back to that in a second. I mean, I think in terms of the Scal is very lightweight and very free and doesn't bind you. And I don't think you need to give the actors a whole lot of input on how to use that system. The only thing we do have to do is the leg comes up fairly high. It's like at a high hip position, the robotic leg. And so we do have to put basically like holster like things on the actors just so they don't put their arm through that. Right. Cause otherwise you just, you wouldn't stop your arm. wouldn't be aware. and also it's cool because when you know that it's there, there's a sort of putting a hand on the hip thing that, that Ardmore does where she's like casually putting her hand on her hip, but she's putting her hand on the robot's hip. Right. Which, so in other words, her hand is floating in space. but we put an object there so she would know where to grab it. The amp suit, I wasn't around for photography on the amp suit on the first movie because I wasn't a production designer. I didn't go down to New Zealand and see any of that stuff. You know, I'm sure that Jim, he did a great job coaching slang on how to be cool about it. They probably had to rehearse and play around with it a little bit, you know, to like, you know, the idea that he pushes a little button, but like in this emphatic way and a little light turns on, you know, to kind of activate the arms. I'm sure that was probably figured out to some extent on the fly, you know, in some. Maybe some poor guy had to figure out how to turn that light on and off on command, because it was just rigged to be on like, crap, you know. So some electrician or ? lighting technician maybe had some stress. I'm not sure. As far as the crab suit, we made the manual knowing that we wouldn't always be there. The people that use crab suits are not the lead actors, right? We don't have like dialogue scenes around the apparatus the way we did with slang on the first movie. And so we did that knowing that we wouldn't be there to kind of encapsulate the knowledge. So we give that to the… assistant director and her department. We give it to our art directors who might be on set, just hoping that the cultural knowledge kind of flows through, you know. ? You know, the way that the movie's cut, it's not some of the subtleties. It would be easy to miss them anyway, but we wanted to have a good system. Part of what we had to think about is the fact that Mr. Krabby Pants has, ? you know, more legs than a person does, right? So in the amp suit, even though we've never really addressed what the hell's going on with the legs in the amp suit in terms of control, The presumption is that there's something that might be analogous to two legs controlling two legs, right? So there's some pedals or there's something down there that you're using to walk around with, which of course doesn't work when you've got four legs on the vehicle. So we came up with the idea that it's kind of like you're riding a creature, right? So imagine that you're riding a horse and you have big robot arms kind of strapped to the front of that horse. So your legs are going to be telling the creature where to go along with some control sticks and stuff that are built into the end effectors, but your arms are directly controlled as opposed to the indirect control. So we built different paddles and levers and stuff into the leg area of the seat so that you actually use like the same way that you might use your knee as to put pressure inside a horse to make it move away from pressure. That's exactly how you do a suit.

Brian Heater (25:25)

Unfortunately, we're coming up on time right now. But looking at your CV, you've worked on all these amazing films with these very iconic directors. ? Three of the big ones that come to mind are the Transformers series, Prometheus, and Avatar. So we're talking about Michael Bay there, Ridley Scott, and James Cameron. Compared to the other two, and this is coming from having just watched these two documentaries, and again, through lying through all of them, is like, James is really involved to a fault with like every part of production. How involved is he with the actual sort of designing and the execution of the like mechanical and robotic part?

Ben Procter (26:11)

I think his level of involvement is always there in super laser detail in the sense of you know looking at the designs carefully and Perceiving whether they feel legitimate and feel real or not, right? Then when that doesn't go right he has to get more involved, you know, I mean we've certainly put designs in front of him where You know, there was certainly moments on Avatar one where like people would get the full, the full war scrutiny into, okay, tell me what this detail on this helicopter does. Like, what is that? What is that? What is that? And I think he did that to kind of shocken all the art department into, Oh God, all right, got to take this seriously. We're working for a frustrated engineer who knows how to fly a helicopter, you know, so you better know what cyclic means and like all this kind of stuff. And so, so he established a standard. I would say that, you know, does he go into the nitty gritty of like every little detail anymore. No, if we're doing our job, he shouldn't have to do that. And he often doesn't. When something is problematic or just because we haven't quite solved it or something that's a high concept element of the script that really needs to get conveyed. Like in other words, this design needs to convey story. And this is true of even the holograms that I mentioned, right? Like he will beat up those holograms just like he would any piece of mechanical design because in some senses, often they're more important to convey to the audience what's happening right now. Like, we're telling the story of how this crazy link system works to the audience and Avatar 1, it has to be simple enough, that's a little cool, but it to be simple enough and clear enough in how it's visually presenting that stuff to communicate clearly to the audience. And so I think he probably comes at it more like that. And I think he's got enough trust in ? me and other nerds that work with such passion in the...live action art department. Fausto de Martini is a name I can mention that Jim is, he loves Fausto. Anyway, Fausto has his hands into many, many, of these designs as we've gone into the sequels. So we have it easy sometimes. If it looks really damn cool and we've clearly done our homework, then it's gonna be okay.

Brian Heater (28:10)

Yeah, well, Dan, it's been an absolute pleasure and I guess we'll see in a few years when Avatar 4 comes out.

Ben Procter (28:16)

Sounds good. All righty. Cool.

Brian Heater (28:21)

Thank you so much to Ben. Thanks to Disney. Thanks to the Avatar team for setting that up. Thanks to Jim. I feel like I can call him Jim now after that conversation, right? Ben. Don't forget to like and subscribe for more robotics content and please subscribe to the Automated newsletter over at automated.fm. And we will see you next week for another episode of Automated.