animoji_apple animoji_apple
Tech

Everything You Need To Know About Apple’s New Animoji

Apple made augmented reality a consumer concept today with its new “animoji” phone feature that allows users to interact with one another as real-time rendered 3D cartoon characters.

Using Apple’s new Face ID hardware, Animoji are animated emoji that use data from the user’s face to create expressions while generating automated lip sync from user voice input.

The looping Animoji can be recorded and sent through Apple’s own text messaging app in iPhone X. The easiest way to understand how it works is to watch the demo:

The underpinning technology behind Apple’s Face ID is the new True Depth camera system that is made up of a dot projector, infrared camera, and flood illuminator, and powered by the A11 Bionic neural engine, which is a dual-core design that performs up to 600 billion operations per second for real-time processing.

That tech, combined with the iPhone’s improved front-facing camera, will be capable of capturing and analyzing 50 different facial muscle movements, and then animating those expressions in a dozen different Animoji.

Apple's new True Depth camera system.
Apple’s new True Depth camera system.

Augmented reality (ar) is not a new technology. And unlike its virtual reality counterpart, which is still hampered by overly complicated hardware, ar has already taken off; face transformation apps, Snapchat filters, and Pokémon Go are all wildly successful examples of the tech. Another big booster of the technology is Facebook CEO Mark Zuckerberg who’s pushing ar as a cornerstone of Facebook’s 10-year plan.

Animoji also has a precedent with real-time character animation tools like Adobe Character Animator and Realallusion’s Crazytalk.

Animoji

Like much else of what Apple does, Animoji is not a revolutionary technology, but Apple’s iteration is a notable moment for augmented reality, as it normalizes communication-as-animated-characters on a mass consumer scale.

With Apple jumping on the train alongside Facebook and Nintendo, it’s safe to say that ar technology isn’t going away anytime soon. As the tech increases in sophistication, it will open up tantalizing possibilities for animators and character designers who may soon be creating animated characters not just for passive viewership, but for people to actively become.

  • Michael Rank

    They’ve brought The Emoji Movie home to us!

  • secretgoldfish

    might be nice to control something like character animator with your phone…….but that would involve adobe and apple working together for the benefit of their customers….so probably as much of a probability as properly/natively tethering an ipad pro to a mac for drawing.

  • waterworld

    Such an irresponsible waste of time and resources.

    • Jim Corrigan

      My thought exactly. Unbelievable.

      • Taco

        Is Apple’s new Animoji app an irresponsible waste of time and resources as a consumer product or gimmick, YES. But as a way to sneak a facial mapping & cloud based deep learning system into every apple fanboys smartphone, it’s pretty clever. Designed to draw from & exploit all users camera data in order to develop future products from Security ID systems to emotion recognition & performance capable AI & much more, all of which they can sell back to the public or both private or government enterprises down the road. You can see where this is all going… ~;^) “harmless fun”

  • AnthonyA

    Hope it works better than their face recognition did for non-white-skinned people.

  • Elijah Samuel Abrams

    At least this is WAAAAAAAY better than The Emoji Movie, the WORST animated feature of this year!

  • J.S

    Emoji movie 2 is looking good

  • Cementimental

    Is there an emoji of Steve Jobs rolling in his grave?

  • mxxx

    kinda cool, oh the life of keeping up with ground breaking creative advancements when you’re a company that isn’t a creative company, good luck to them.

  • Chris Hatfield

    Apple acquired Faceshift a few years ago. This is the outcome of the integration. Prior to shutting down all commercial distribution of Faceshift, you could indeed control character animation. It was a badass trackerless motion capture software. https://web.archive.org/web/20150905072832/http://www.faceshift.com/#2
    https://techcrunch.com/2015/11/24/apple-faceshift/