|Barba proposed that first unit run a set of witness cameras during live action photography that would be synced to the main system. Doug Roble, winner of an Academy Scientific Achievement Award for tracking software, rewrote DD’s tracking code to utilize the data from the witness cams. “It was still an incredible amount of labor for the tracking team, a huge hurdle that scared everybody because it had never been done before.”|
The lighting needed particular attention. “The human head,” said Barba, “is extremely sensitive to color and light, and Claudio Miranda’s photography is so beautiful. Our digital heads had to work with the way he lit the set, the way the eye highlights hit and the subtleties they strove for on set. We had to devise a system that would give us more fidelity than we’ve had previously, so wrote a whole new system of rendering to take that data and make it work with the skin. We spent months writing Mental ray shaders and a slew of tools to take all the on-set data from traditional techniques like HDR’s and set surveys, and apply that to shaders and geometry to rebuild 3D sets that were physically accurate, had the right bounce and falloff and color temperature, so that our head always existed as if it were in that set at the same time it was being shot. That was a huge process, day to day during principle photography, to feed all that information back to a system we were building here at the same time.”
|On a sound stage in Burbank, while watching select scenes, Pitt stood motionless in the center of the stage surrounded by four Viper cameras in a semicircle while he performed the part of Benjamin. There were no markers on his face, no need for tracking data. Image analysis technology was used to match those shots of Pitt performing to the appropriate background footage, deriving timings of, for example, when an eyebrow or lip moved. DD then mapped out the timings on Pitts face. Character Supervisor Steve Preeg’s team developed code that took the rough data and searched the relational database for shapes that best match the various facial components on a frame by frame basis to reconstruct a 3D GC head. The head motion was taken from the on-location actor.|
Animation Lead Matthias Wittmann, whom Ulbrich rightly referred to as the “genius animator”, was one of only 6 animators on a team of 150 artists. “I was in charge of testing the blend shapes from the Mova scan, of talking to the modelers to see what they could keep and lose, what was the neutral or zero position of the head. For example, a smile. When our rig is smiling, the whole face is somehow deforming. We took into account every millimeter on the face of the real Brad, determining if a deformation was due to the muscles driving the smile, or was it due to, for example, him raising his eyebrows? If you have too much deformation on every part of the face and you mix those with other shapes you get errors of strange deformations happening on other sections of the head. We did this to every single shape.”
|Matthias Wittman, Animation Lead.|
|DD put together a plan using optical flow and machine vision cameras and doing what the Mova Contour system offered. This led them to a facial system based off of Pitts’ face and not off of skin and muscle systems that are traditionally used, and the riggers best guess, a method that allowed them to build a sophisticated rig that only moved like Pitt. As blend shapes were defined, a sophisticated system gradually emerged. The blend shapes used for Pitts face totaled 180, though more were built in.|
“That is not a lot, but if they are well developed you don’t need more,” Wittmann explained. “Our eye system was tricky. The animators just needed to move the eyes around and the blend shapes drove the movement around the eyes. Some techniques were added to the head to detect shape animation rotations, and would dial in certain neck deformations and wrinkles under the jaw. There were systems built to animate Benjamin’s tongue and his adams apple when he swallows.
|“We thought we would need way more animators. We started with three, and built a pipeline for them that made it really easy to get all the reference images of Brad from the witness cameras on set. All this was in sync and the sound was there, and all they had to press one button and the scene was set up and they could start animating, and click one button to send it on to lighting. The animation was really handy and convenient.” By setting up such a pipeline, they team could pick and choose their best talent without worry of their technical prowess. |
“We figured we would be famous after this, or they would chase us out of the country! It’s thrilling working on this not knowing. I’m really happy all the departments were so high class on this, everything from tracking to lighting to compositing, everything was so good and you didn’t feel you were wasting your time because somebody was messing it up.”
|THE END OR THE BEGINNING |
“There were so many parts to make it seamless,” Eric Barba exclaimed. “So much had to happen during the live action shoot to get the lighting to work. The team writing tools for lighting had to get it right so that head felt like it belonged there. And even if it felt like it belonged there, if it didn’t feel alive and didn’t emote, and it didn’t make you laugh, then it wouldn’t work. That is a huge credit to the animation team, and Steve Preeg and his team of riggers and animators who built that system so that this character becomes believable, the tracking, to get that head perfectly on that spine so that you buy that it exists. And to get the paint and roto work just right, and the loving last bit of artistry which is the composite.
|On the tug boat heading for Russia, the first time we see Pitt as Button, only with makeup.|
|“There is a sensitivity in compositing, and Janelle Croshaw, Paul Lambert and Sonja Burchard, our compositing sups, did an amazing job keeping that consistent through our 325 shots. Once someone works on human skin you realize you have to have such a fine sensitivity to color. The skin falls apart very easily if it gets mooshed around. Our color temperature process of taking in the correct balance of on set lighting, and adjusting the HDR imagery and reapplying it, that whole pipe was built by some really talented people so when the renders came out all the color temperature was correct and the compositors didn’t have to moosh around too much.|
“All those pieces have to be there. The eyes, the eyeglasses, the teeth, the tongue, the lips, the moisture, the hair, the right type of light, it all has to work or it all falls apart. There are so many people involved with each one of those shots. That is what I am most proud of, I am most proud of the team.”
|WHERE'S BRAD PITT?|
The baby is an animatronic puppet. The first time the audience sees Benjamin is when he is a toddler eating at the table, playing with his food. For the next 52 minutes in the movie it’s all the CG head spliced on to other bodies.
The CG Brad Pitt is handed off to the real Pitt in makeup, who remains in the film until he returns as a child again, a feat handled by Lola in a process Fincher called “youthenizing”. Lola also made the Daisy character, played by Cate Blanchett, appear younger.
Pitt, in makeup, is first seen when he is on the tug boat heading to Russia, when Benjamin is sitting on the side of the tugboat reading postcards from Daisy. Brad Pitt was on that site and on that ship. When he saw the movie, he said to Fincher ‘I don’t remember shooting that’. It was because it wasn’t Pitt, it was the CG head on another actor. Said VFX Executive Producer Ed Ulbrich, “That was very satisfying, because if we can fool him…”
|This film could have the first all CG haircut in a film. In an extreme close-up on the back of Benjamin’s head. His character spans 35 years of life in this movie and so needed some 60 hair styles. |
“We had a screening up at ILM and up until that time I was still a bit nervous, but there is a group of people up there working on something similar. One of them asked, ‘what was my favorite shot?’ and when I told them, they looked at each other and said, ‘that was CG too?’ and I knew that we were in good shape.” -- Eric Barba, VFX Supervisor, Digital Domain.
Ironically, one of the hurdles was finding animators. “We started hiring, thinking everyone would want to work on this movie. It’s Brad Pitt and David Fincher and facial performance that had never been done before! But people find themselves cleaning up motion capture curves or that they don’t have time to go into detail, or they are pushing around deformers that are so slow you are 20 years older after the project. It was hard to get any animator interested!” - Matthias Wittmann, Animation Lead.