Disney Researchers Have Developed An AI Facial Recognition Tool For Animated Characters
A team of researchers at Disney has revealed details about the Content Genome, a new AI tool it developed to recognizes faces in the company’s animation archive.
What are the computer scientists at Disney doing exactly? Engineers and scientists at Disney’s Direct-to-Consumer & International Organization (DTCI) have developed a platform that can automatically identify faces, alongside other elements like locations, across all the content in Disney’s archive. In a Medium post, the team explained how they harnessed AI machine learning framework Pytorch to develop the tool.
How was it made? Facial recognition tools for photography and live-action footage — in other words, for real human faces — are already well established. But Disney’s archive contains a lot of animation, and animated faces are another matter. They are inevitably stylized, often extremely so: think of the cars in Pixar’s Cars, which have human facial features but in barely human proportions.
The DTCI team was tasked with adapting existing software to spot animated faces in its archive. They describe the challenges in their post. For example, because animated backgrounds are often relatively simple, the software initially read anything that stands out against them as a face. With training, the Content Genome was taught to recognize faces reliably, and can now also recognize other easily identifiable visual features, like locations.
What’s the point? The Medium post presents the Content Genome, which has been in development since 2016, within the context of Disney’s shift to digital distribution. As the article suggests, the recognition tool — combined with appropriate tagging — could make search functions and recommendation features on, say, Disney+ more effective.
The software could also be used in-house by artists who want to track down past examples, for reference, of a specific type of design or effect. Similarly, executives could employ it for “business intelligence,” as the post puts it: they will now have a clearer insight into the elements that recur across their most popular works.
What next? The team wants to keep developing the Content Genome and teach it new tricks. Visual elements like faces are one thing; more abstract concepts are a new challenge.
Anthony Accardo, director of r&d at DTCI, told Engadget that they want to harness “multimodal” machine learning techniques — in other words, to get the tool to interpret and combine multiple sources of information within an image, and use this context to develop a deeper “understanding” of what’s going on. But, he adds, that will be difficult.
The bottom line: Machine learning is becoming an increasingly integral part of how content is made and promoted. Disney is at the vanguard of this trend.
(Image at top: recognizing the face of Bunga, from Disney Junior’s “The Lion Guard.”)