Awesome Animated 3D Illusion on iPhone

David O’Reilly has built a very cool animated walk cycle that takes advantage of the iPhone’s motion sensitivity.

O’Reilly describes the effect on his blog:

“The application works by assuming a constant viewing angle (35-45 degrees), typical for when the device is placed on a tabletop. The 3d scene’s perspective is warped using anamorphosis, the same technique used in Hans Holbein’s painting The Ambassadors. This application does the exact same but updates dynamically.”

There’s been some controversy online about whether O’Reilly’s animation is actually motion-sensitive or if all the animation was completed earlier and he’s simply moving the iPhone to match the onscreen action. Regardless, the reality is that there is amazing potential for interactive cartoons on the iPhone and other motion-sensitive devices. Let’s do a little blue-sky thinking and imagine the possibilities. Instead of simply watching a cartoon, viewers can now interact and control the actions of their favorite characters. A simple tilt of your iPhone could send a character walking in any direction. A quick shake could make your character turn away from another character. Don’t feel like watching an 11-minute cartoon today? Control the pace of short and make it a four-minute cartoon. New technologies will open up new narrative possibilities for animation artists.

The linear cartoon is so 20th century. For a new generation of kids, watching a cartoon with only one ending (i.e. every cartoon today) will test the limits of their patience. It’ll be the equivalent of riding a horse-and-buggy after cars had been invented. Sure, Chuck Jones and Mike Maltese came up with a good ending for One Froggy Evening, but today’s cartoonists can come up with twenty different endings for their shorts, exploring all sorts of what-if scenarios. They can begin to understand their creations from a deeper, more psychologically complex perspective. As a viewer, if you like a particular ending, you can control your character’s actions to always achieve the same result. But every individual viewer can also change the outcome of the cartoons they watch with a simple tilt or turn of their screen. Viewers can become engaged in the universe of their favorite cartoons as never before, and it will become a much richer experience for both creator and viewer. All of this could happen, but it will take the combined efforts of programmers, animators and studios with the vision and desire to push their cartoon characters into the 21st century.

Previous Brew posts about David O’Reilly HERE, HERE and HERE.

Read more CGI

  • vineet

    tech really does blow one’s mind to the possibilites for storytelling, but wasnt that post a description of a video game?

  • http://tarasis.net Robert McGovern

    Certainly not impossible, there is a dice app for the iPhone called Mach Dice by Mach Kobayashi who appears to work for Pixar. At the end of his demo video he shows a 3d effect on the dice by gently tipping the phone around.

    http://www.machwerx.com/ for his site, video there too.

  • http://www.j3d.com.au Josh

    Wow! that is super cool. I wonder if it would be possible to have the iPhone as the viewing window and be able to view 3d scenes using it. Probably doesn’t have the power though. Does the iPhone have GPS?

  • Chris L

    That guy is consistently impressive. As for the doubters, it seems like it would be pretty difficult to choreograph moving the phone to sync with the action, especially since there didn’t seem to be much of a beat to it. I think it’s the real deal.

    As for the future possibilities of animation, it seems like you’re talking about a more linear version of what video games already (try to) do today. I can’t find the link, (for all I know it was posted here), but some university techies are developing a piece of software that allows you scrub through videos by grabbing elements of a video and moving them around the screen (eg. a ball being thrown). Of course there is only forward and backwards, but a combination of little tricks like these could lead to some pretty interesting things.

    Instead, I’ll settle for this: http://www.youtube.com/watch?v=XLikgptMD98
    3D motion capture graffiti.

  • amid

    Robert: Those dice are great. Thanks for pointing it out.

    Vineet & Chris L – You’re both right. What I’m suggesting certainly bears similarities to videogames. As the different media converge, they’re all going to share a lot of the same elements, such as interactivity.

    The difference with what I’m proposing however is to use interactivity for purely narrative purposes and as a way of advancing the traditional cartoon form. Games usually incorporate narrative within a larger context, either trying to solve a puzzle or scoring points or something. With interactive cartoons, the choices made will advance the story, allow viewers to see new gags and explore the relationships between characters.

    Another significant difference is that when you play a Wii or Xbox, the screen itself is stationary. If you flip your TV around, the onscreen objects wouldn’t be responsive to that movement. With handheld devices that have motion-sensitivity, the rotation and movement of the screen will determine the behavior of characters. It’s a huge advance that should be explored by cartoon creators. The next SpongeBob or Shrek could very well be an interactive creation.

  • slowtiger

    Although I’m impressed by the technique I don’t believe it will revolutionize the medium. Stereoscopic film, nowadays called hologram? Old hat, and artistically rarely impressive. Interactive cartoons? I’ve seen interactive films since the 80′s and none of it was worth the effort. And since especially animation made cutting edges an art form I doubt they will change their attitude and produce 20 endings for one cartoon.

    Only in games this will make sense.

  • http://checkeredgeekcartoons.blogspot.com Zach

    To be fair, the Wii is a motion sensitive gaming console, which could achieve similar effects to what we saw in David O’Reilly’s video. What he did with the iPhone though is much more direct, and a bit more intriguing.

  • http://gagaman.blogspot.com The Gagaman

    Amazing blue sky thinking of yours Amid, maybe you could call it a “video game”. I kid, I kid. =)

    In other news that rather funky 3D walking cat kind of reminds me of Octocat: http://uk.youtube.com/watch?v=8hm3E2cGQE4

  • Harv

    Today’s kids are bored to death listening to sermons in church, too. Imagine the twenty-one different endings they might morph onto the end of any given religious spiel. Church will be as potentially entertaining as a video game, high praise indeed.

  • http://www.rauchbrothers.com Tim Rauch

    Linear storytelling began with Adam and Eve and will die in the nuclear holocaust… but it’s good to have other things happen along the way.

  • Vintage Season

    Very cool… but David O’Reilly’s own site followed up the initial announcement with the following:

    23/08/08

    “The iHologram has become one of the All time most popular videos on Vimeo in the space of a day, and is currently on the front page of almost every technology site… I’m getting way too many emails about it, so for those who had to know:

    “The iHologram app was not real. It was an illustration of an idea I had which I believe could work with the technology (combining anamorphosis and motion sensing). Unfortunately I’m just an ideas person, I can show how things should look, but I’m no hardcore programmer.

    “I’d be happy to collaborate with a developer or studio who want’s to make it happen, I’m bursting with ideas for the interactive world, but right now all my attention is on filmmaking.

    “My aim with this was to tackle the problem of 3d viewing in an original way using current technology, not fool anyone… so for those who doubted but still supported it, respect. I hope it inspires some talented programmers out there.”

  • doug holverson

    Anybody know the song playing during this demo?

  • http://rauchbrothers.com Mike Rauch

    Much agreed, Tim. Not to say that new possibilities in storytelling aren’t intriguing, but new technology isn’t going to change a human tradition that stretches to the beginning of humankind.

  • The Animator

    That’s pretty sweet makes me almost want an iPhone, almost.

  • Eimhin Mc N.

    The whole interactive movie idea, be it cartoon or live-action short/ feature, is an interesting one.
    Art is about the artist making choices, deciding what it is they are trying to say with their work and focusing the content of the work towards those ends. Usually what constitutes poor art is when these choices are not made, everything is shown or given equal importance such that the viewer cannot draw the meaning forth.

    Interactive movies are relient not just on the ‘director’ to construct the text for viewing but also on the viewer to construct his version of the text upon his viewing of it. (This concept in itself is not old, everyone will perceive a piece of art, or indeed anything they are exposed to, differently than anyone else.) What makes it ‘new’ is the extent to which the viewers’ experience or personality can exert control over the text, even beyond what the original artist intended, in this fashion it is approaching certain aspects of found art or open-source programming.

    The main crux upon which interactive film-making relies is the director’s willingness to give up control over their ‘film’, to intentionally make the narrative muddy, indistinct, so that the viewer doesn’t just reveal the meaning but constructs it themselves.

    This means that interactive movies can go one of two ways:

    The First: The Film is indistinct, it is a kit-box to make films, where the Viewer, through successive attempts adapts and learns how to ‘play’ the new game, forming numerous stories and meanings out of the narrative components, or building blocks.
    OR:

    The Second: Film makers refine their control to such an extent that the viewers are offered the illusion of choice, no matter how many alternatives or divergent choices the make the film still converges upon the same meaning and intent that was inherent in the Director’s ‘version’ of the film.
    It’s obvious from this that you’d never really have true interaction in this context, one body must exert control or impose limitations upon another, as in video games, where you ‘perform’ in the narrative but must progress with it. For there to be true interactivity there must be equality among those involved.

    (Taking a bit of a side step and perhaps a sommersault or two, to a somewhat related subject)
    A good example of an interactive-reality (if we make the assumption that film is trying to portray a reality through it’s narrative and mise-en-scene) is the online videogame/lifestyle choice Second Life where the participants have almost equal footing with one another, and all conspire towards building a world. I by no means endorse it, or even consider it up to a comparison with the concept of what an interactive film could or should be, but it is terribly successful at what it does, and seems to have droves of converts and supporters flying to its forums day in and out.
    My big problem is that it is just replicating the messiness and chaos of life, there is no focus, or intent to it. It is not saying anything about life, it’s not communicating meaning, it is just doing.
    This too would be my concern for interactive movies, that they would lose their worth as objets de Arte, simply become novelty, like watching The Wizard of Oz with Dark Side of the Moon.
    Of course there’s always the chance that an interactive film, with all it’s myriad of components and choices, will fall into the lap of a viewer who has the ability and singular qualities to shape a film from the formless clay, but the likely hood is that they’ll feel limited by what’s on offer, either in the subtleties or the grand scope.

    I think interactive film to be a bit of a dead end, or at least not an end in itself, but a progression between things. It will not allow either the director or player to fully control the film, so both will become frustrated.

    At the moment, there are already film-makers coming out which have manufactured films from assets made available to them, either through programs, equipment or the vast array of instructional and theoretical material, now more readily available than ever. They have bypassed limitations such as cost and convention because their work is being created wholly by them, with no preconceived notions of what’s expected of them, but just what they wish to create.
    Technologies like these are putting tools in the hands of everyone, and there will no doubt be surprises in store on the front of interactivity, but there must be a distinction made between playing and creating, in the sense of narrative: to play there must be rules and to create there must be control.

    This is my opinion on the matter anyways, though it would be cool if we could somehow bypass this and open whole new avenues of experience.

    I hope I haven’t ranted on too much, and that I make some sense.

  • http://UncleEddiesTheoryCorner Eddie Fitzgerald

    WHAT THE HECK!?! That’s incredible!!! How did he DO that!? I had no idea that was possible in 2008! If he cheated and animated the turn-around to make it look holographic, I’ll still doff my hat for the glimpse into the future he gave us. If this can’t be done now, you know it can’t be far off.

  • autisticanimator

    One idea for a holographic mini-theatire could be that because the viewier has more access to the background than the character in the movie, they can notice something in the background that the character can’t and then could turn the device so the character either can’t get to that side of the background or heads directly torwards it…

  • amid

    Awesome idea autisticanimator. The possibilities for new types of storytelling are limitless when the user can control the direction and movement of the screen they’re using!

  • http://rossphillips.net rp

    turns out its fake!
    still, the idea is what counts.

  • Chuck R.

    Amid is correct in that there is a future in interactive animation, but the other comments are even more correct in that it already exists in the gaming world. What Autistic Animator proposes could be two years down the road, but it will extend from the gamers’ evolutionary line.

    As Eimhin Mc N. astutely projects, anything interactive from the cinema world will be gimmicky, and actually hurt filmmaking more than help it. In fact, I’m surprised that Amid is even suggesting that a populist approach to moviemaking will result in better stories. Hollywood has tried multiple endings that the populace can choose —it’s called the focus-group. There have been other faddish attempts to take creative control away from artists, leading to such inspiring works of high art as the inane “happenings” of the 60′s, and the movie “Clue” with it’s wonderful multiple endings. Once the entertainment industry decided that script-writing was too bourgeois for modern audiences, we got game shows and reality TV, culminating in “American Idol” —the bastard child of both. I know the populistic screed looks great on paper, but like any form of populism, you have to take responsibility for what’s shat out.

    What Amid needs to understand about film is that it’s an artform for fascists. Cinema requires it’s visionary to be a control-freak, and it provides the perfect, controlled viewing atmosphere. Animation is the most fascist of all, because every frame is planned, boarded, drawn and checked.

    Other popular artforms are challenged in ways that cinema is not. Our feelings toward music are colored by where we are when we first hear it. TV and the internet (our most populist mediums) are prone to all kinds of distractions. In contrast, Cinema is so controlled. The director’s got you for 2 hours of uninterrupted time —commercials are over before the lights go out. He knows how comfortable your seat is, he knows the exact environment —cool dark room with nothing but a big screen. The viewer gets to choose butter or no butter. that’s it. That’s the way it should be. We pay eight dollars to put ourselves in the hands of a visionary with something to say. If they fail, we leave the theatre, get on the internet and dump on them. This is a good thing my friend, and good artists take advantage of it.

    History’s best filmmakers connect with wide audiences, but let’s not kid ourselves: the best of them are 1 part Hans Christian Andersen, and two parts Francisco Franco. Walt Disney, Orson Wells, Chuck Jones, Ralph Bakshi, George Lucas, Peter Jackson and Michael Moore — despots all of them.

  • amid

    Chuck: Prognosticating the future requires creativity and imagination. Explaining why a car can’t drive to the moon doesn’t mean another type of vehicle, like a spaceship, can’t take you there. Similarly explaining what works with a static screen and linear storytelling has little relevance to the ideas I’m proposing in this post.

  • Christina S.

    All this talk of interactive storytelling reminds me of a video game called Chrono Trigger, actually. While it indeed involved certain gameplay elements that, well, made it a game, there were interesting narrative choices that made it extra special. For example, the beginning of the game takes place in a fair, where you can participate in several activities to pass the time while waiting for a friend to get ready. One of the things I did was eat a bit of meat lying on a concession stand, thinking “Well, no one’s telling me to not steal it, so it’s okay for me to take it.” Turned out that incident came to bite me in the butt later on, when I was in court and the concession stand owner was on the witness stand, antagonizing me for stealing the food!

    For people who don’t play video games a whole lot, let me tell you something about them. Normally in a game, people don’t care if you break into their house, steal all their treasure, and ruin their lawn. It’s rare that a video game would actually reward you for not being a jerk without TELLING you in the first place not to be a jerk.

    Additionally, while I haven’t beaten the game yet, I know that there are multiple endings as well. While it’s still a game, with no motion-sensitivity or holographic animation, story-wise it sounds pretty close to what you imagined, Amid.

  • Josh

    There is an interesting game in the pipeline for the PS3 called “Heavy Rain” which is probably the closest thing I’ve seen that could be classified as an interactive story. If they could use the Wii technique of putting on a pair of glasses to allow head tracking in this game then it’d probably be the closest thing I can think of to what people are imagining in these posts.

    Read about it and see the video here:
    http://www.joystiq.com/2008/08/21/joystiq-eyes-on-heavy-rain/

  • matt

    Ahh, does no-one remember the “choose your own adventure” books from the 80s? Happy memories…

    Also, I’m sure Don and his Dragon’s Lair lawyers will try to do something here – God knows they’ve wrung every possible cent outta that one for decades!

    On a more serious note, surely MMORPGs have already taken the element of choice/participation beyond this dual concept to something more akin to a social art project? Yes mostly the organic and unplanned possibilities are wasted when the participants wilfully choose repetitive and myopic stereotypes, but the theory has definitely been taken further than what is being discussed here.

    Also there were a couple of Cd-games on the Sega genesis that attempted to do this on-rails interactive kind of storytelling decades ago (whose names I can’t recall – there were execrable though). I think it’s interesting though that after all this time many people are still talking about the future being one or the other, when the present shows that it’s already BOTH passive AND interactive storytelling, co-existing.

    Finally, in terms of perception/interraction/progression even puzzle games seem to be in on the act – I’ve been playing something called Echochrome which is basically M.C. Escher meets videogames. It might be a bit tenuous, but you change your viewpoint in 3d space like our little cat up there to reveal or conceal elements, and change your perception between 2d and 3d to get through to the end, if that makes sense.

    P.S. RP, where’s the info on the fakery?

    I don’t see why this can’t work in theory, as it’s relatively simple in terms of accessing the correct frame either from a pre-existing/rendered database or rendered on the fly based on feedback from the accelerometers (although generating the frames would only really make sense in 3d/cg rather than hand-drawn animation). Be interested to see the fake way to do it.
    P.P.S. Amid what you originally commented on sounds a lot like duelling directors! ;)

  • http://www.kohrtoons.com Rob K.

    I find the concept of interactive animation a tad fun. Although I think that it is already being nearly achieved though almost any modern video game. That said I totally agree with Matt I think that a non-linear story where you pick the path may hearken too much to those books where you picked your scenario. Not to say it can’t be done, but that it ruins the whole concept of an artist presenting his vision.

    That said the fakery info is here:
    http://www.macrumors.com/iphone/2008/08/22/ihologram-looks-remarkable-but-is-it-real/

    The reason it can’t be done has to do with two issues. First the effect can only be achieved from on particular angle, so the moment you move from that angle the illusion is ruined.

    Also the iPhone cannot detect “rotation”, for rotation awareness it would need something like a compass in it, or some other way for it do distinguish a change of origin. It can detect angle, but angle is relative to gravity. If you want to see something funny try Monkey Ball or another ball app in a car, then slam on the breaks, the ball will jump because of the forward momentum the accelerometer still carries.

  • http://www.warrenleonhardt.com warren

    hearken too much to those books where you picked your scenario. Not to say it can’t be done, but that it ruins the whole concept of an artist presenting his vision.

    Well, as a kid I LOVED those books. A movie/cartoon version of a similar set up would be fantastic way to kill some time when you’re tired of gaming for a while. Ruins a vision? Not if it’s part of the vision. Fake or not right now, it’s a great concept worth pursuing! Nice post.