pearl_allumette_invasion pearl_allumette_invasion

Observations from the VR Front: ‘Pearl,’ Invasion!’ and ‘Allumette’

Immersive this. Virtual reality that. Interactivity and narrative immersion have emerged as the new frontier of animation, and along the way, they’ve become buzzwords for everything exciting and cool.

Trying to understand these new approaches to animation was why I checked out the Tribeca Film Festival’s virtual reality showcases a few week ago. The festival’s Virtual Arcade, the name for the second half of Tribeca’s admirably extensive interactive programming, proved useful in furthering my understanding of current narrative approaches in vr animation.

This piece will discuss three high-profile projects that had their world premieres at Tribeca: Pearl, the latest in Google’s Spotlight Stories initiative; Invasion!, from the Silicon Valley start-up Baobab Studios whose creative head Eric Darnell directed Dreamworks’ Antz and Madagascar; and Allumette, from another Valley start-up Penrose Studios.

My first insight from Tribeca is simple but important: no one is actually talking about the same thing when they use the terms ‘interactive,’ ‘virtual reality,’ or ‘immersive.’ Each of these ideas means something different in the current ‘Wild West’ atmosphere of vr production. I experienced well over a half-dozen animation projects at Tribeca, and each had its own approach to immersion and interaction, its own ideas of what it wanted its audience to experience.

About the only concept that virtual reality creators agree upon at this point is that projects will cede control of the camera to the user, but even that core principle is iterated in different ways throughout projects.

The most sophisticated narrative among the three projects was Pearl by Patrick Osborne, the ex-Disney animator who directed the Oscar-winning short Feast. It was made for Google’s Spotlight Stories series, whose original mission upon launch in 2013 was to develop interactive mobile experiences. While the Spotlight Stories still produce mobile versions, Pearl also has separate vr versions for both Google Cardboard and HTC Vive headsets (the latter of which incorporates positional tracking, offering the ability for users to move around in the car rather than observe everything from a fixed position in the front passenger seat).

Pearl was the only one among the three that had so much happening in its 360-degree environment that it made me want to watch it again to see what I’d missed. Its concept — telling a story while in a moving car — was especially well suited to the medium as it created a natural reason for continual action to happen around the viewer. Combined with the frequent cutting (a new innovation — and significant technical hurdle — for Spotlight Stories), the project kept me engaged throughout.

One of the things that the Spotlight Stories have done since the first project — Jan Pinkava’s Windy Day — is to present stories without a set running time. For example, if you follow the main action, your experience might last five minutes, but when you explore the environment, the experience could stretch out to, say, six or seven minutes. I expected that this temporal plasticity would become a basic component of virtual reality storytelling, but many vr projects, including Invasion! and Allumette, have fixed durations. The Spotlight team understands that the temporal pacing of vr storytelling isn’t that of film’s, and experiences benefit when the viewing time is as fluid as the ability to explore one’s surroundings.

Invasion! is directed by Dreamworks and PDI veteran Eric Darnell. Pitting aliens against bunnies, the short offered the least interactivity among the three pieces. The majority (if not all) of the action takes place on a 180-degree stage; in other words, while you can look around the 360-degree space, nothing happens behind you except for static scenery, so the viewer has an incentive to look straight ahead. Also, the viewer (you are a stand-in for one of the bunnies) remains in a fixed position so there is little chance of missing key action.

By limiting the variables, like range of movement and staging, something interesting happened that hasn’t happened in my other vr experiences: I chuckled at some of the humor. Comedy is one of the more difficult things to pull off in virtual reality because so much of it is reliant on timing and staging, elements that a director concedes to the viewer in vr.

Not that comedy is impossible in vr, it’s just that no one has been interested enough to explore the possibilities yet (and there are indeed amazing possibilities when I think about it myself). At this current juncture though, Darnell’s approach suggests that limiting the viewer’s capacity to interact with the environment can help increase the comedic value in a vr experience.

Allumette, from Penrose Studios (which has raised $8.5 million in seed funding), was among the most talked-about pieces at Tribeca. Wired went into hyperbole overdrive when it called the project “the first vr film masterpiece,” but the project is impressive for illustrating the potential directions that can be taken by vr filmmaking.

There’s a crucial distinction between the first two projects discussed above and Allumette. Whereas Pearl and Invasion! lock the viewer in the center of the stage, and allow you to look outward, Allumette puts the viewer on the outside, allowing you to look in on the scene.

Allumette’s story unfolds on a single set — a bridge that is part of a floating Venetian sky-town. The short makes elegant use of the HTC Vive’s positional tracking feature, allowing a viewer to not just look around, but to also move in and out of the scene. For example, as you watch the main characters interact, you can move closer into the scene and see the characters in detail or move further away to get an overview of the setting. You can duck your head under a bridge and look at the action from the other side of the bridge. You can stage the action in an infinite number of ways.

Most impressive, when you move your head close to the docked airship, you pop into the interior of the ship itself and can wander through its rooms. It’s a fascinating effect that reveals new locations (and narratives) as you move around.

Here’s where it gets interesting: should you choose to look inside the ship, there will still be an animated story happening on the outside, and it doesn’t wait for you. This opens up the intriguing possibility of simultaneous narratives being told in vr, making it all but impossible for a viewer to take in the entire experience in one viewing. The different paths a viewer could take, and how those paths affect their understanding of the story, is what could ultimately separate vr from traditional linear narratives.

Allumette is not that however. It suggests multiple narratives, but never follows through on that promise, making it more of a technical gimmick than anything. The twenty-minute project is paced with such lackadaisical indifference that you won’t miss much should you choose to explore your surroundings rather than focus on the story (which is what I ended up doing to alleviate the boredom).

Further, looking at the action from different perspectives reveals nothing new to the viewer. Its story, like traditional film, is still designed to be experienced from one perspective (and one perspective only); the chance to reveal new information from different angles was a missed opportunity. (Just to offer a simple example of what I mean, imagine that two characters are having a conversation, but if you look at them from another angle, one of them is hiding a knife behind his back. By virtue of handing over the camera to the user, vr offers a unique opportunity to reveal narrative detail to users based on how they view the story.)

But it’s not hard to see why Allumette is attracting so much attention. For years, CG features have shown us three-dimensional characters from only one perspective. Why continue doing the same thing in vr? Allumette proposes that if your world is three-dimensional, then the audience should be able to experience it from any angle — and even experience multiple concurrent stories.

Being an animation historian, there’s always the temptation for me to identify parallels between the development of new mediums and animation’s historical development. If we were to identify where we’re at in the progress of virtual reality, compared to the animation art form’s development, I’d suggest that we’re currently in the Farmer Alfalfa and Krazy Kat phase.

VR hasn’t yet found its Walt Disney, the individual who’s going to push the craft and medium to new heights. And it probably won’t find that person for a while.

Chances are that vr’s Disney isn’t someone who’s currently working in the industry. The real innovations in vr storytelling, I believe, will come from those who are currently in their teens or younger, in other words, those who have grown up with a digital screen as an extension of their self, and who have indulged in the complex narrative immersion of gaming universes from near-birth.

The upcoming generation of vr creators will, of course, build upon these foundational vr experiments, just as Disney himself built on the work of veteran cartoonists-turned-animators like Winsor McCay, J.R. Bray, and Max Fleischer. In the meantime, projects like Pearl, Invasion!, and Allumette will continue to develop and search for an authentic vr language. There’s a long way to go yet.

  • ea

    What awards are VR projects eligible for: Oscars? Emmys? DICE or other video game awards?

    • Last year Shannon Tindle’s “On Ice” was nominated for an Annie for best animated short.

  • ddrazen

    First off, I was disappointed that “Allumette” was not based on the picture book of the same name by Tomi Ungerer, which was his take on Hans Christian Andersen’s “Little Match Girl.” I still think it would make a great short film.

    “Invasion,” though pretty much nailed to the floor by the viewer’s POV, is able to overcome that by engaging the viewer through the characters making eye contact. It was a trademark move of animator Tissa David: having her characters make eye contact with the audience, however briefly.

  • VR makers (especially those from a film background) would do well I think to study games, because while VR clearly has some unique problems and possibilities of its own many of the questions it poses regarding interactivity, dynamically changing stories, pacing and staging have been, if not outright solved, then at least thoroughly explored in the interactive medium of videogames.

    Maybe start here for interactive animation:

    Then read up on some level design.

    • Taco

      I agree, VR is an amazing step into an undecided future involving animation, games & interactive media. And for games & adult entertainment, you can imagine those products & markets will be well catered for with VR. But for filmmakers & artists, will VR be as easy to embrace practically? Or will we grapple with interpreting the new fundamentals of the VR format? As coming to terms with a VR world won’t change the fundamental design & choice related shot structure of good narrative & character based story telling. Interactivity & entire world/screen immersion sounds great, but I think there will be a clear divergence between the formats of VR & “Film” based content, due to all the additional variables to be taken into account when producing a VR-Worlds-Content, while also trying to tell a concise & meaningful story involving those variables. It will be a balance between VR-Viewer-Freedom & the readability of the actual VR-World-Content you want the participant to grasp or experience. There is no clear shot economy or clear visual staging & readability in an open & free-form VR world. Most practical VR story telling will probably be “on rails”, guiding the viewer in a specific way and cutting off their choices in certain ways like game designers do. While VR does have the potential to make every movie you watch at home into a full scope IMAX-type experience, there are many logistic hang ups for a VR “film” to be practical in telling stories or giving the experience(s) we know & expect with films today. This is both a subjective PRO & a CON for creators. You can’t listen to and take in a story the same way you do while sitting around a camp fire VS while you’re out hunting & gathering, or physically involved in an activity. These are different forms of mental engagement. I’m not saying one is better than or shouldn’t be explored more than the other, but in nutshell: VR is an immersive & interactive experience, while film is a passive but cognitive engagement in a designed & intended story sequence. Think of reading a comic book VS playing a video game. Unless it’s your job to be both audience & character finding clues or involved in a sequence like a chase, either running from or going after X or Y, (which is a video game) I can only see VR & Film as two different formats, with different end goals for how an audience is intended to process the information. A films information is presented, while a video games information is explored. Film, Comics & Books (words with no pictures) all have a level of clear communicated intent. Games & VR favour experiential choice & can only have a similar communicated intent if you are on rails or boxed into an experience. Games & VR are fascinating because they add an element of discovery & a level of haphazardness to the world & all the elements you are presented with. Films & Books are a more direct & dictated engagement with another persons mind/view, rather than an engagement with nature & and environment. They are both emotionally satisfying & tangible in different ways. The problem is the type of expression & viewer experience you want to instil VS the format used to deliver the intended experience to the viewer. And there’s an element of Occam’s razor here when it comes down to what’s necessary to tell an indented story. I think to solve some of the complexity problems that come with the sheer scale of full 360 degree VR immersion; you could simply scale down the content to a more readable, but still immersive & interactive desktop or a VR story book size experience. Rather than being about the format, our art is about the content & cognition of that content. The more things change, the more the cognitive limitations of human perception remain the same.

    • For some recommendations, I suggest looking at games like the Mass Effect series, The Witcher 3: Wild Hunt, Gone Home, Firewatch and The Last of Us for some ways to tackle interactive story and characters.

  • Taco

    “Games & VR (…) can only have a similar communicated intent if you are on rails or boxed into an experience.” Wow, you certainly abridged those 6 paragraphs & my overall sentiment well, didn’t you? Ninja, I agree with you and am excited for VR and all the possibilities & discovery. I’m simply stating that there are fundamental differences between the formats. Pros & Cons with interactive & non interactive content design. Apples & Pineapples. See the above.

    • I don’t disagree with many of your other points, was just responding to that specific observation.