Oscar shortlist films run gamut of visual effects storytelling

Dec. 31, 2019 1:26 PM PT

Remember when visual effects were mostly associated with explosions, superpowers and other adventures some respected directors might not consider “cinema”? Now, even those filmmakers are taking advantage of the storytelling doors modern technology is opening.

Not teaching old icons new tricks

Martin Scorsese’s $159 million “The Irishman” moves back and forth over decades to arrive at a somber meditation on regret. Previously, that might have required 30- or 40-something-year-old actors to play the characters over so much time, either with old-age makeup or with other, older actors cast for the story’s later years. Instead, advances in digital de-aging allowed septuagenarian acting greats Robert De Niro, Joe Pesci and Al Pacino to play their characters throughout the entire film.

“We ended up rethinking the whole de-aging thing,” said VFX supervisor Pablo Helman, noting what emphasis Scorsese put on conveying the performances. “We had to make sure we had a way to capture all the nuances without markers [the tracking dots applied all over actors, including their faces, for performance capture] because the actors did not want to go that way. Marty said, ‘If you solve this puzzle, it’s going to be something everybody in the industry will pick up so actors won’t be burdened by the technology.”

With Industrial Light and Magic, cinematographer Rodrigo Prieto and the Arri Group, the filmmakers developed a new kind of rig -- a “three-headed monster” that flanked the main camera with two infrared cameras.

“We capture every movement in the pores, in the skin. Every one of those pixels in the face becomes a marker. It creates what we call ‘geometry’ – a 3-D rendering of the face. It’s very nuanced because now you can see all the movement around the mouth and the eyelids. You can see the reverberations of the consonants – Ts and Ps and Fs and things like that – through the face. We’re trying to get very, very subtle performances.

How “The Irishman” cracked the performance-capture code for a trio of acting greats and a legendary director.

Beyond the initial task of peeling decades off the performers, the VFX team also had to stay true to the characters, rather than, perhaps, viewers’ expectations.

“For the shots of De Niro in the war, at his youngest, late 20s, he didn’t want to see ‘Taxi Driver,” said Helman. “Marty didn’t want to go back 30 years in the story and see a Jimmy Conway from ‘Goodfellas.’ He wanted to see a younger version of Frank Sheeran, and that meant – he was 6’4”, 240 pounds, he had a wider face.

“The advancement is allowing the actors to say, ‘I don’t have to deal with all this technology; I can do what I was hired to do.’ ”

The taste of Will Smith

Two-time Oscar-winning director Ang Lee took on a different age-related challenge with “Gemini Man.” His action film pits one of the best-known movie stars in the world, 51-year-old Will Smith, against a 20-something ... Will Smith. Sort of.

“Junior,” the younger version of Smith, isn’t a de-aged motion-capture performance by the actor; it’s a totally synthetic, digital human. That meant, for instance, stunt performers and stand-ins didn’t have to hide their faces.

“We told the stunt people, ignore the camera, just act,” said VFX supervisor Guy Williams. “So we have all these beautiful shots where we come right up on ‘Will Smith’ right after he’s ridden a motorcycle.”

The technology pioneered by Weta Digital for “The Lord of the Rings” had already proved audiences could accept entirely digital characters. “Gemini Man” VFX supervisor Guy Williams said they moved the process forward by, for instance, “rewriting the way we dealt with skin — treating skin as colors that occur at different depths.” The problem, he says, was Will Smith.

“It’s insane how much a human being can recognize another human being at a glance,” he said of how well viewers knew the actor. “The slightest change to a person’s face as he starts to smile, or blink, or look to the left or whatever, it blows away your ability to recognize the person.

“Early on, what we were told was, ‘I believe it’s a real person, it just doesn’t look like Will Smith. It looks like his cousin.’”

This look at the making of a scene from Paramount’s “Gemini Man” shows how the filmmakers took elements of Will Smith’s performance to general a completely digital character, “Junior,” in his place.

In the finished scene, “Junior” has an emotional moment.

Raising the bar even higher were the film’s 4K resolution, high frame rate (120 frames per second), stereoscopic capture and lingering close-ups — all unforgiving of trickery. After many trials and errors, their worries were allayed when they put the finishing touches on a shot early in production in which the two Smiths glance at each other after surviving an explosion:

“Everybody said, ‘Holy crap, it looks like Will Smith, feels like Will Smith, tastes like Will Smith.’”

Star Wars: The forced perspective

The “Star Wars” franchise reflects a four-decade history of visual effects. But while the series’ second trilogy relied perhaps too heavily on computer-generated images, the third trilogy returned to practical, or at least practical-looking, effects where possible.

“I thought doing a retro movie was the wrong thing to do,” said “Star Wars: The Rise of Skywalker” VFX supervisor Roger Guyett. “What we wanted was to make a contemporary interpretation that had that same feeling you had in the original movies that was very tangible. That meant going to the locations. We tried to capture as much on camera as possible, but the reality is, you still end up with close to 2,000 [VFX] shots.”

The final trailer for “Star Wars: The Rise of Skywalker” shows off some of its top-shelf visual effects.

So apart from using advanced digital effects to make titanic space battles believable, Episodes VII-IX employed, for instance, animatronics (think “Empire Strikes Back” Yoda vs. “Revenge of the Sith” Yoda). The popular character Maz Kanata, for instance, is a puppet rather than a digital creation, Guyett said. And when it came to destroying a planet, “kind of as a nod to the original movie and Alderaan blowing up,” he said, they detonated a physical miniature and used forced perspective for other miniatures on good old Tatooine.

One of the most important tasks, Guyett said, required digital wizardry to realize a hybrid performance: the late Carrie Fisher’s final screen appearance.

“Essentially, we took outtakes [from ‘The Last Jedi’], then we built a digital Carrie Fisher outside of her face. So what we see in the movie is really a digital character with a live-action face.

“It’s like a magic trick, that idea of distracting the audience. We’re using her face, and that’s where people are looking. We just changed everything else — hairstyle ... jewelry ... costume. We didn’t want it to look like we’d just taken outtakes from the previous movie; all the fans would be going, ‘Oh right, that’s the outfit she wore in blah blah blah.’ It’s incredibly complicated, without getting into the weeds of it.”

The screenwriters “took all the lines she’d spoken and built the script around those lines so we didn’t have to change what she said,” said Guyett. “There was an integrity to that performance.”

The end of ‘Infinity’

Then there was “Avengers: Endgame.” From the perhaps-you-missed-it tidbits of the heroes’ time-travel suits being digital to the transformations of Skinny Tony and Old Steve; from the nuanced, synthetic Thanos and Smart Hulk to one of the most colossal scenes of combat onscreen, the culmination of Marvel’s “Infinity Saga” ran the gamut.

For “Avengers: Endgame,” the VFX wizards had to pull out “every trick in the book.”

“It wasn’t enough to have the big battle. Joe Russo came to me and said, ‘The [first two] ‘Avengers’ movies had these long shots that connected different heroes during the battle ... we wanted to do it in this one. It has to be the oner [extended shot] that ends all oners,’ ” said VFX supervisor Dan DeLeeuw.

“You’ve got 20 or so heroes, including the Avengers, running across the stage, and it wasn’t big enough for everyone. We had to pull all the set dressing. You added tens of thousands of digi heroes and villains ... pretty much everything you’ve seen in earlier Marvel movies, running at each other. And then, the sky battle and Giant Man punching a leviathan, and that transitions into Black Panther’s moment and then Drax has a moment and then Iron Man and Rescue have a moment.

“It took three months to work on the shot; 90 different artists worked on it.”

As in “The Irishman,” however, some of these VFX wizards’ proudest moments came in conveying the complexities of performances — such as through the extreme aging of Steve Rogers/Captain America (Chris Evans).

“It was about keeping enough of Chris in the performance so you could recognize the character you’d spent so many films with. ... We cast an old-age double so Chris performs the scene, then the old-age double performs the scene. Then we take the skin texture from the old-age double and remap it onto Chris, so you can keep Chris’ eyes and smile so you know you’re seeing Chris Evans in there.”