Fahrenheit Digital

Still suffering from jetlag… I force myself to to go bed at around 4AM. Just my luck that every day I have a course that I want to attend that starts at 8:30AM.

Started the day with a half-day course titled “Face to Face.” Started out with a bang; the facial capture & retargeting of “King Kong” was detailed. Initially I had thought that Weta was using a muscle system for Kongs face. Instead they basically used the Andy Serkis facial capture to drive a series large number of well-built blendshapes. But because the human facial musculature is slightly different than that of a gorilla, coupled with a different set of expression that these two species can create, Weta developed a method to retarget the human facial muscles & expressions onto the face of their mythical silverback gorilla.

The retargeting was based on the Facial Action Coding System, or FACS. This system breaks down the expressions of the human face into a series of defined muscle movements. Although a similar system exists for chimps, Weta had to create this FACS for their gorilla.

The results were absolutely amazing! The subtelty that they were able to capture and retarget was astounding. Hands down the best performance capture I have ever seen. The fidelity of their capture was so precise that they were even able to reproduce the most subtle eye moves through tracking the smallest corneal bulge during the capture sessions.

Hats off to Weta and a great performance by Andy Serkis!

Other parts of this talk included a presentation by George Burshokov, a former coworker of mine from ESC who now works at Electronic Arts. During the Matrix sequels George was instrumental in developing what was called “Universal Capture,” a way of capturing subtle facial performances as well as incredibly high resolution texture information simultaneously… all without using any markers.

George and others at EA have refined this method so that it works to create realtime facial animation. The data sets are much smaller and he used markers, but the facials expressions they are able to produce for the Tiger Woods game he demo’d were fantastic. Definitely a technology that will add to the realism and fun of the next generation of videogames.

One reason they chose to explore this method over hand keyframing of the faces is the “Uncanny Valley.” A Japanese robotocist, Masahiro Mori, surmised that humans find synthetic reproductions of human likenesses more appealing and experience a higher degree of empathy… up to a point. Once these synthetic humans reach a level very close to “real” in their appearance the degree of human appeal and empathy plummets significantly. Hence, the Uncanny Valley.

So as videogames and motion pictures strive to create realistic humans we are finding ourselves on the cusp of the precipice of this Uncanny Valley. Some would argue that our industry has fallen off the edge a time or two!

George believes that using performance capture to more accurately reproduce the detail and subteleties of the human face seems to be a promising way to climb out of the Valley. His results on the game and during production of the Matrix sequels were phenomenal.

Later, I jumped into a talk on content creation but decided to depart once it was apparent that the topic was not what I had initially expected it to be. So I met up with some friends for lunch, took a nap then woke up in time to go to one of the parties happening that night.

I decided not to go, so I walked out for dinner but the place I wanted to go (Barking Crab) had a long line of people streaming out the door. Went back to my hotel, into the restaurant there and ran into more friends (one great thing about Siggraph is that you get a chance to meet up with your old friends regardless of where in the world they are working). We had a great time!

After dinner I met two men from an animation studio in Tokyo. My wife is from Japan and we visit her country about once a year. I have been lucky to have had the opportunity to visit a few schools and studios while there.

A friend working at this particular studio asked if I would take the time and meet with his coworkers, which I was more than happy to do. I was dreading the fact that my nihongo is terrible, but I figured between my terrible Japanese and their better English we could manage. Well, it turned out to be a fun time and they were both great guys. We spoke at length about the animation industry in Japan and the changes it is going through right now.

Eventually, several of my coworkers from Disney joined us and, since they are all nice folks, we all had a fun time just hanging out and talking.

This entry was posted on Friday, August 4th, 2006 at 12:30 am and is filed under Industry News. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.