What did you say, Face?

by Rob Pullins | December 26, 2011

The television series “Lie to Me” depicts a Dr. Cal Lightman (Tim Roth) as the leader of a firm hired by third parties to find “the truth.” In the show, Dr. Lightman uses the Facial Action Coding System, microexpressions, and body language to figure out what people are really thinking and, of course, know when they are lying. Through this show many have become aware of Dr. Paul Ekman, the real world inspiration behind the show, and his pioneering work in the field of technology psychology called facial emotional measurement.

Dr. Ekman has been credited with developing the Facial Action Coding System (FAC), which according to Fast Company is “… a comprehensive dictionary of facial expression measurements that has become the scientific underpinning for human observer and automated facial analysis around the world, [and] across academic and commercial fields.” With this new added insight into human subconscious through their unconscious reactions and behaviors, along with the development of a machine to assist with the microexpression analysis, it wasn’t long before marketers sat up and took notice.

According to Harvard Business School professor Gerald Altman, 95% of all cognition occurs in the subconscious mind. It is subconscious cognition that influences buying decisions, which is what marketers and advertisers are often trying to connect with using marketing materials, including visual media like video or graphic or Web design.

Large corporations have already begun to use the developing practices of monitoring brain patterns of consumers when they view their ads or products, a practice known as “neuromarketing.” Now they are looking to FAC to serve the same purpose: to get an understanding of viewers’ subconscious reactions to their ads and products by their facial expression. Neuromarketing techniques are believed by many to be a much more accurate way to evaluate emotion than using focus groups because focus group members often inadvertently react to outside influences rather than stating their true feelings.

A somewhat similar practice has been happening in Web design for some time with eye-tracking. While not getting into the realm of emotion, eye-tracking has enabled Web designers to better understand what features, colors, placements, or movements draw the attention of a viewer and the path through a Web site that their eyes will take automatically. Using FAC, some firms are looking to take this insight into Web design one step further.

E-Storm blogger Ivana Petraskova wrote about a software being developed by Affectiva to “measure emotion in web video.” Using a Webcam along with the software Affectiva is able to analyze facial expressions related to happiness, but there is no intent to stop there. As they continue to develop the technology they hope to be able to detect a range of emotions, from happiness to boredom to annoyance. While this technology began with the intent of better understanding individuals with communication challenges, such as autism, it was only a matter of time before the avenue of marketing potential was pursued as well. You can try out their software here if you have a Webcam.

Facial emotional analysis has also found its way into the world of digital animation. According to FastWeb, Pixar’s Toy Story was the first movie to use FACS, and since then Ekman has also worked on Rise of the Planet of the Apes and Avatar, as well as with Disney to add more believability to characters’ expressions. He has, of course, also advised the TV series Lie to Me.

The possibilities and effects of using FACS and a better understanding facial expressions are only just beginning to develop. Aside from the marketing and animation applications there is a potential social impact. As mentioned before, Affictiva’s original goal for their software was to overcome communication challenges. There are also hopes by some, according to FastWeb, that this same technology can be used to detect drunkenness, depression, intent to kill, or even disease.

Rob Pullins is a new media marketer and world traveler. In 2013, Rob was Managing Editor of NOD while he was Director of Marketing at Sessions College.

This blog is powered by Sessions College, the leading online school of visual arts.