New Tales from the Borderlands features cinematic performances thanks to Gearbox's use of motion capture (or mocap for short). This is the first Borderlands game to feature full performance capture, recreating each actors' body motions, facial expressions, and spoken dialogue all at once! The result: stunning visuals and a deeper narrative experience.

Amélie Brouillette, Associate Producer at Gearbox Studio Quebec, oversaw much of the mocap work for the game as the Shoot Manager (cameras, not guns) across 11 months and a whopping 700 hours of filming.

To learn more about the painstaking work that went into bringing the characters of New Tales to life, we sat down with Amélie.

METICULOUS MOTION CAPTURE

Previous Borderlands games used mocap to bolster their animations, but that was solely reserved for capturing body movements, with actors dubbing in their dialogue later on. For facial animations, a program was used to match the mouth movements to the audio, and everything else had to be keyframed (i.e. animated by hand in a labor-intensive process).

But for New Tales, full performance capture was a literal game-changer. Accurately capturing body movements, detailed facial expressions, and audio as one complete package gave the Gearbox team the tools they needed to create a more immersive story.

But even with all that sweet, sweet data, the team still had to overcome technical limitations to bring the game to life.

"We could only have four facial cameras at the same time, so we had to prioritize what to shoot and when," says Amélie. "Each recording session has to be planned out in super intense detail. Does this scene have props? Will an actor touch something? If yes, what, when, and where? What's the height of that object? Will they look at something in the world, like a picture hanging on the wall? Where is the picture, and how big is it, and what is it a picture of?"

Save for any props present, the motion capture stage is a blank canvas. To help ground the actors in the scene, the animation team provided animated storyboards and layouts for each shot.

"These help the actors know where they should be, what they're doing in a given scene, and where the cameras are," says Amélie. "It's a rough blocking with all of the characters moving like robots, but it lets us know where they are and what they should be mindful of given the way the camera is framing the shot."

On top of all that, time is of the essence when shooting the segments of a scene as the system can only handle two minutes of data at a time.

Amélie estimates that roughly 98% of the game's roles used full performance capture, with some exceptions being the extras who populate crowded scenes and characters like Sponsorbot that are purely digital with no corporeal form to capture.

Once shooting wrapped on each episode the best takes were selected. Those were given to the animation team who worked to bring it into the game, polish the motion, and heighten movements to create the final experience.

ONLY IN THE BORDERLANDS

The Borderlands present some interesting challenges for full performance capture. The universe is full of danger and Mayhem, as exemplified by Tediore crony Brock, a talking gun with legs.

"We actually recorded both body motion and audio for Brock—we had a prop of Brock's gun body made of wood, and the actor was moving it and talking at the same time," laughs Amélie. "It was awesome. I really like Brock!"

Because some characters don't have faces, like assassin bot L0U13 and Tediore soldiers with full helmets, the performers and animation team had to employ different tactics to create emotions, including body language and screen displays.

The Gameplay Trailer above shows towering Eridian Guardians who wreak havoc on the Tediore forces that dare disturb their Vault.

"That Eridian shoot, my goodness... it was one of the most intense days for the mocap team," Amélie laughs. "We hired a stunt worker who specializes in stilts, with the director for mocap and the director of animation on hand to figure out how to capture the Eridians' movements in just the right way.

"That scene has people dying everywhere, including the Tediore soldier who flies through the air and lands spine-first on a spike. That was actually mocapped, if you can believe it. We had a guy—safely—jump backwards onto a pole with a mattress on it.

"I think for that one scene alone, we shot for an entire day."

With four possible responses at multiple points in the script, the team had to plan carefully around branching narrative paths and multiple conversation options.

"Usually, we would record the question from the main path, then one of the choices that we think is most difficult in terms of continuity," says Amélie. "Then we record the final part of the answer after those questions, constantly referring to a tool we used to chart out the many paths that the script takes.

"For the actors it was really challenging at the beginning, but after a day or two they had it down. Usually we shot four or five sequences in a row, then returned to the choices we didn't shoot initially.

"When going back to the alternate choices, we had to be very careful about a lot of things, but two in particular. First was the continuity: if, when you ask the question, your hands are in the air, then you move them during Choice A, you have to reset the position of your hands when shooting Choice B, and so on.

"Secondly, the same rule applied to props. If during the question you had a gun in your back pocket, and during the answer you pull it out, then each time you start with the question, you have to start from the same position of having the gun in your back pocket.

"I'm usually more of a macro-manager, but in those cases I had to very much be a micro-manager!"

OVERCOMING NEW CHALLENGES

Perhaps the biggest challenge of all was having to shoot during a global pandemic.

"It was really tough working in this then-unknown area," says Amélie. "I had to work on two-thirds of the shoots from home with five screens in front of me, across a three-hour time zone difference, working with the team in Vancouver from here in Quebec.

"There was a screen for the script, one for a visual of all the actors, a window for chatting with the designers, writers, and animators, and so on. It felt like I was in a spaceship," she laughs.

Numerous steps were taken to ensure the continuous health and safety of the cast and crew, with everyone taking COVID tests every 48 hours. Scenes were also reworked and rewritten to fit safety guidelines.

"I even hired body doubles that were on call if any of the primary actors got sick. Fortunately, the planets aligned and none of them got COVID-19," says Amélie. "It was definitely a more complex plan; we shot with a very small-scale crew and avoided contact whenever we could."

Aside from the COVID protocols, Amélie and the team also took care to make sure the actors weren't drained by having to record a barrage of tough scenes.

"When I created the schedule, I added a highlight to denote the more emotional scenes," Amélie explains. "During those scenes, you don't rush. You take your time; you're looking at the actors, seeing if they need a break and if they're OK.

"What was amazing with that project is: the actors said that typically their contracts for shoots span one or two weeks, rarely a full year. Over the course of our 11 months of shooting, we really had a chance to develop a working relationship. They could come to me and say 'Amélie, can I take five minutes just to calm down?' or stretch, or the like, because it's really intense for them.

"They were super professional and always gave their best."

You can take the time to appreciate the magnitude of mocap work that went into New Tales from the Borderlands when it launches this Friday, October 21. We can't wait for you to experience this extraordinary cinematic adventure where you'll Make Mayhem Your Business!

Check out the other New Tales from the Borderlands Dev Diary articles: New Tales Dev Diary #1 - The making of your underdog story

New Tales Dev Diary #2 - The People of Promethea and Tediore