• ere are the scary facts about
    Sony’s animated feature ‘Monster
    House’. Directing the $75 million
    dollar production was Gil Kenan’s
    first gig after graduating with a Masters degree from UCLA.

    Sony Pictures Imageworks captured the performances of all the actors and the dialog for all the human characters in the film while Kenan directed them on stage. It’s a horror comedy animation with a PG-13 rating, the first CG movie to enter the spooky film genre.

    The risks proved worth taking. ‘Monster House’ opened in second place behind Pirates with a $22 million plus weekend. And, some critics have uttered the “O” word (for Oscar). “I was really in the right place at the right time,” says Kenan. Winning UCLA’s Spotlight award with his short film “The Lark,” didn’t hurt either.
    “The Lark,” mixed live action pixelated actors and a stop motion puppet over 2D paintings that Kenan hand-animated over live action reference plates. “I created a ridiculously arcane system to torture myself,” he says. “I ended up doing a one-man version of the ‘Tron’ production in my kitchen.” The award-winning student film, among those screened at the Director’s Guild, caught the eye of an agent’s assistant at CAA and the powerful agency signed him up. “I was really, really shocked,” Kenan says. “I was convinced I would make animated films in my kitchen for the foreseeable future.”

    After that, Kenan made the rounds. “I was having three or four meetings a day,” he says. “And then one fateful day, I met Robert Zemekis.” Zemekis and co-producer Steven Spielberg had been considering producing Monster House, a film about a house that comes alive. Coincidentally, Kenan’s film, “The Lark” takes place inside a house, and in the film, Kenan began to explore how a house could have personality. “I started to breathe life into the house,” he says.
    So, Kenan read the ‘Monster House’ script and met with Zemekis and Steven Spielberg. “It’s crazy isn’t it?” he says. “I brought in a ton of drawings I had done and talked their ears off.” Zemekis had directed and produced The Polar Express, for which Sony Pictures ImageWorks pioneered the studio’s ImageMotion performance capture system.

    ‘Monster House’ used an updated version of that system. And it was a good fit for Kenan. “This way of making animated films is still in its infancy in some ways,” he says. “The crew, until Imageworks came on, was a live action crew and to them it was strange and mysterious waters. You can get creative sparks that way when you put people into situations they’re not familiar with.”
    The production starts with the performance capture. “Imagine a theater in the round,” says Jay Redd, visual effects supervisor at Imageworks. The motion capture stage was actually a 20 x 20 foot space, 16 feet tall surrounded with 200 infrared cameras and six video cameras. Each actor wore between 60 and 80 reflective markers on their bodies and a similar number on their faces. “Usually we had our three kids and one or two adults in any scene,” says Kenan. “The actors could look at each other and physically give life to their performances. In many ways, it was what you hope to get with great voice recording, but when you put actors together, you get what happens between the lines,” he says. “The looks that you’d lose with voice acting alone. That’s the stuff I’m excited about.”
    “I find that the kind of animation I respond to in terms of humans is exactly what I got from the performance capture,” Kenan adds. “I respond to films where performances aren’t quite as broad as the American mode of keyframe animation. This yielded subtle performances with lots of gray areas within every line.” Kenan used the video reference taken during the motion capture sessions to cut a reference version of the film that had all the actors’ performances, but, of course, no backgrounds, sets or props, which Imageworks would add later.

    Meanwhile, Imageworks created the stylized 3D characters in Maya, which resembled but didn’t match the human actors. “Because the characters didn’t look exactly like the actors, we had the flexibility and freedom to push, pull, and exaggerate to accentuate expressions and actions,” says T. Dan Hofstedt, co-animation supervisor with Troy Saliba. “We kept the texture and the style of the motion capture and added to it to create a hybrid performance.”
    For example, Hofstedt explains, the scary old man Nebbercracker who lives at the Monster House, doesn’t look like Steve Buscemi, who performed and voiced the character, but the spirit of Buscemi’s performance shows through. An “integration” team at Imageworks took the first stab at feeding the actors’ performances into the characters. In this first stage, the integration team applied the body data from the performances Kenan had selected for the film to the CG characters - no fingers, no faces at this stage. “This is where they addressed the differences between the actor and the physical puppet,” says Saliba.

    Imageworks then sent the data-driven CG characters back to Kenan. At this stage, Kenan had an entire film with all the CG characters performing on a virtual stage. So, the next step was to shoot the movie. To make it possible to design camera moves, Imageworks provided a system they call “wheels.” Kenan spent four months working with a live action Director of Photography shooting the virtual film in the “wheels” room. The DP used standard film equipment modified so that the cameras would directly drive a virtual camera. If the DP panned the real camera, the scene on the 50-inch plasma screen moved accordingly. “The director could put the camera at any angle and use that to break down scenes into shots,” says Saliba.
      Next page
  • Once Imageworks’ data integrators had the camera angles and shots, they refined the body motion and added data from the facial capture. Although data captured from the physical performance drove the CG character body, Imageworks treated the facial capture data differently. Using the Facial Action Coding System (FACS), developed by Ekman and Friesen in 1978 to isolate facial muscle groups triggered by emotions, Imageworks’ Mark Sagar adapted the studio’s facial animation system so that the motion capture data would trigger combinations of FACS shapes. Imageworks had each actor try to make the 80-plus FACS shapes - an inner brow raise, a cheek raise, nose wrinkle, cheek puff, and so forth. Then, modelers tried to duplicate those shapes. “Hopefully, the shapes on the characters kind of matched how the actor did that shape,” says Hofstedt.
    To test the shapes, animators tried matching video reference of the actors. “We wanted to see how far we could go in caricaturing the expressions.” For a final test, they captured each actor saying a standard line with a range of emotions. “That was our first mocap joining,” Hofstedt says. The animators then went back and forth with the modelers who tweaked the shapes - making a mouth wider, eyelids bigger and so forth - to find the right balance before handing the characters to the 60 animators who would create the final performances. “To me, the motion capture represents live action reference,” says Saliba. “We could use it to emulate reality or to add depth of performance and still go stylized and exaggerated.”

    “Motion capture doesn’t make animation cheaper or faster”, he adds. “It took longer and cost more money. But, I think we got a texture and depth we couldn’t have gotten with keyframing and the director was interested in that.” Kenan came to the film admittedly biased against CG. “I had this built-in animosity toward fully CG imagery,” he says, “because I feel it lacks a tangible relationship to the audience.”
    “There’s a veneer that exists from 3D renders that puts me at a distance,” he says. “I like things to feel tangible. I like seeing hand-painted textures, thumbprints in things. I had to work very hard to create a system where I would feel excited by the result.” Thus, capturing data from live-action actors helped Kenan mimic the performance quality of stop-motion animation. A new rendering system helped Imageworks mimic the look. “CG is beautiful,” says Redd, “but we wanted the antithesis. “It’s been a dream of mind to make a stop-motion film in CG. To make the film feel tangible. I wanted it to look like it was made of something real.”

    To do that, he wanted to use such techniques as global illumination, radiosity, ray tracing, bounce lights, and indirect diffusion. With a background in photography and cinematography, Redd, in fact, wanted to light the film as if he were lighting it on set. “I wanted the color of the carpet to affect the scene,” he says. Marcos Fajardo made that possible. The author of a rendering system called 'Arnold' helped Imageworks adapt that system for production use.
    “We were using global illumination and indirect diffuse from the get go,” says Redd. “It influenced the look of everything we did. We often talk about getting more light, but for this film, we didn’t want to see everything.”

    “We could reveal what we wanted through lighting tricks,” he says. “Something might have a highlight, but we don’t see what it’s made of.” The studio’s internal compositing software 'Bonsai', helped compositors play further tricks with optical effects, atmospheric effects, and a little fogging.The scary lighting was particularly important in the monstrous house. With 40,000 animation controls, the house became one of the largest characters animated at Imageworks, rivaling the train in Polar Express.

    “The house is how I justify the work that it took to create the human characters and bring life to them,” says Kenan. “This is a story that could only be told as an animated story and the reason for that is the house. It’s a living breathing character. We had to create a house that didn’t tear the fabric of the reality of the story.”

    In addition to characters, Imageworks also created all the backgrounds. The childrens’ houses are fully decorated, the suburban neighborhood is filled with 3D models backed with matte paintings and encircled with a 3D sky dome. Imageworks typically uses 'Cinema 4D' to paint textures for projections and Maxon’s 'Body Paint' for other textures.

    Some trees are sprites rendered with Imageworks’ 'Splat' rendered, but the trees near the house are fully rigged 3D models. Particle systems and rigid body dynamics in Side Effects’ 'Houdini' helped the crew tear things up. “This is like a little disaster movie on one level,” says Redd. “We have fire, water, serious mayhem. Cement cracks, walls implode.” 'Houdini' also helped the crew procedurally create and distribute thousands of toys.“The mixing of all these techniques is very exciting,” says Redd. “It’s an interesting way to make a movie.”

    And, it’s an interesting movie. 'Monster House'’ has helped extend the genre of films using CG animation, and the film’s success should offer the young director new opportunities. “I would love to see more weird uses for CG and 2D animation,” he says. “Ones that push the idea of it beyond the standard vehicle for family comedy.”

    Monster House
    Sony Pictures Imageworks
    Facial Action Coding System
    Previous page More CGS Articles

blog comments powered by Disqus