When Oren Jacob left his post as CTO at Pixar to start ToyTalk, it wasn’t clear what kinds of projects the new company would undertake. Along with co-founder Martin Reddy, who formerly led software development teams at Pixar, Jacob hinted at interactive apps that kids could communicate with back and forth. Now, the company has released its first project: The Winston Show, an iPad app that falls somewhere between a game and an interactive television show. Aimed at kids around the ages of 6 and 7, the app uses press-and-hold technology to allow players to talk to the game’s characters and receive answers in real time.
In an interview with TechCrunch, Jacob said that he was inspired by watching children talk back to their televisions while watching Blue’s Clues and other shows where characters consistently break the fourth wall. The ToyTalk team took an approach similar to the process of creating an animated film: voice talent, animators, modelers and writers were assembled under the guidance of the company’s director, Bobby Podesta, also a former Pixar employee.
During the process, Podesta led the team in developing an automated lip syncing technique based on speech recognition that will accommodate the thousands of lines of dialogue each character must have to respond to the user’s verbal input. “With more than six hours of content in the app, we knew it would be too much dialog to animate all the lip sync by hand,” Podesta told Digital Media World. “With this in mind, we developed a way to take the show’s character dialog, extract the phonemes from the audio performance, and assign mouth shapes that would then be animated programatically.”