Annecy: Google Spotlight Stories Releases ‘Piggy,’ A New Approach To VR Interaction

A common comment made by those viewing vr or 360-degree films is that while watching one part of the action they fear they are missing another part of the story that might be happening behind them. The latest Google Spotlight Stories film – well, it’s more of an experiment – is aimed at addressing that concern.

Piggy, directed by Jan Pinkava and Mark Oftedal, and released this week at Annecy 2018, is a vr experience where the viewer has a role in determining part of the story. As you watch an animated pig considering whether he might indulge in a whole cake, your head and eye movements determine – to a degree – what the pig does.

That means Piggy’s filmmakers needed to develop real-time techniques to correspond with viewer head and eye movement while continuing to showcase what appears to be a normal animated short. Cartoon Brew caught up with Pinkava at Annecy 2018 to talk about why Piggy was made as an experiment, how it was produced, and where the director wants to take this immersive technology next.

Whereas the majority of Google Spotlight Stories that have come before have been short films, Piggy is intentionally an experiment — an experiment as a response to the way people have been viewing vr films. “In this case,” said Pinkava, “we really wanted to do a show that had an appropriate idea for pushing interactivity as much as we could, or take a big step in it.”

The pig character, who appears in the film against a simple white background, is still a fully realized animated character – i.e. it is not a video game character controlled in any real-time-like manner – but it does interact with the viewer. “For this film,” said Pinkava, “we asked, ‘How far can we go with a fully animated character that is still interactive?’ We wanted to really refine  things that we hadn’t quite fully fleshed out before so that we could pay very close attention to whether you’re looking at Piggy or the cake that he’s lusting after, or not at either of them, and the timing at which that’s happening.”

New technologies implemented for Piggy made that possible. This included what Pinkava describes as ‘view-driven interaction’ that involves placing targets in the scene that are essentially areas that, if you look at them, something else will occur. So if you turn your head towards the targets, or move your head or your device while you are watching Piggy, different things will happen in the film.

“We have an underlying engine that works with our story editor,” said Pinkava. “The designer of the show can create a graph of all the different animations and parts and transition between them based on triggers of looking or not looking at one place or another. It matters to Piggy, during the film, whether or not you see him eat the cake.”

A further part of the interactive element of Piggy is the eye contact with the character. Piggy is designed to look directly into the viewer’s eyeballs – not over your shoulder or at your side, but directly at you as if he somehow recognizes you. To do that, the team created a way for Piggy’s eyes to move in real-time to point at the viewer. “In 6-degree-of-freedom virtual reality where you can walk around the room, the eyelines change in real-time to line up with you,” said Pinkava. “So that means that you get that eye contact, you get that feeling of connection, and you believe and understand and see that Piggy is seeing you and reacting to you.”

Then there is the sound and music of Piggy, another part of the interactive side of the vr experience. Most of the time, the film features a song from one of Schubert’s song cycles, with the words slightly altered so that the German language voice is actually singing about cakes and pigs. It was arranged by sound designer Scot Stafford. “Scot just added this as a temp track,” said Pinkava. “He said, ‘What about this?’ Everyone started laughing. So I said, ‘We’ve got to go with it, it’s hilarious.’”

“What’s great about that particular music track as well,” added Pinkava, “is that it’s rhythmic, which is why Scot chose it to go with Piggy running around. The way it’s structured within the story editor system that we have is that it’s interruptible. Multiple times in the show you’re interrupting Piggy as you discover him about to have to cake, and the music itself is structured in pieces so that there are points at which you can effectively interrupt it very well for comedic effect.”

Piggy’s sound mix also includes the pitter patter of the character’s trotters where necessary, moving around the listener’s field of hearing. But even here, the interactive elements continued. “We do this thing,” said Pinkava, “where sometimes Piggy is just off screen, just where you’re not looking, and you can hear him, but he’s not actually there. So if you turn your head to see him, he’s gone. He’s trying to lose you, he’d rather you weren’t looking at him. So there’s a couple of places in the show where the sound design is telling he’s somewhere where he isn’t and it’s another way of trying to have an interaction with the audience through sound that allows you to have a – we hope – funny experience.”

With each new project, Google Spotlight Stories has ultimately been exploring new stories and new storytelling methods. Piggy is aimed, says, Pinkava, at being a jumping off point to perhaps a more complicated film with that same kind of interactivity where the viewer feels slightly more in control.

“We now have the means to do something that before you couldn’t do so easily – this idea that you can set up the system where the interaction is well-defined and you can polish it and really tweak it,” said Pinkava. Piggy is such a simple idea but it relies on, for its comedic effect, that the timing works and we’re letting go of a lot of timing. That’s the risk in these films. Comedy is timing, and in this case timing is sort of up to the audience.”

“Piggy” is available now on the Google Spotlight Stories App for iOS and Android, on Daydream, Steam, and Viveport.