Hacking the Kinect

Microsoft’s new controller-free gaming environment called Kinect for XBox 360 also doubles as a powerful digital toy for hackers. The open source drivers on the Kinect allow users to hook it up to PCs and push it in different directions like this:

The above was created by manipulating Kinect data in realtime through a C++ coding platform called Cinder. The end result makes live-action footage look like it’s gone through a Michelin Man toon shader.

Here is another person who is using the Kinect to create 3-D space:

Of course, when I look at any of this, I immediately think of the possible filmmaking and animation applications. It’s not what’s being done with it at this moment, but what could potentially be done with it tomorrow. And clearly, I’m not the only one who’s thinking of those possibilities. This video shows how an off-the-shelf motion control system can be used to recreate virtual 3-D cameras:

Just as Flash democratized drawn animation and made traditional animation production accessible to the masses, technologies like gaming motion-control systems are democratizing the high-end SFX and CG industries. The next generation of young animators won’t just be making Flash cartoons in their bedrooms; they’ll also be producing short films with special effects on a par with Avatar and Harry Potter.

UPDATE: Ken Priebe pointed out this digital bird puppet created with Kinect. More details about their process on Vimeo:

(via Kottke)


  • http://www.fleischfilm.com Thorsten Fleisch

    I like this much more than the linklater treatment.

  • NC

    QUICK!! Take it down before Robert Zemekis sees this!

  • http://beaudetteblog.blogspot.com Grant Beaudette

    I’ve kinda assumed that someday you’ll be able to buy a cheap webcam that comes with a mocap setup the way you can get one with green screen software now. This looks like a first step.

  • http://www.frankpanucci.com Frank Panucci

    That dysmorphic thing kills! What the heck is it doing with the data? What kind of data does the Kinect gather anyway?

    • Oluseyi

      The Kinect has a video camera, an infrared camera and an infrared emitter that spreads a stable scatter pattern across a room. By looking at the deformations in the positions of each dot, it can surmise the shape of the surface interrupting it.

      So the data you get from the Kinect is like a heightmap that describes surface topology. The dysmorphic demo is taking that data as a starting point and then extruding/imploding surfaces from there (it looks like it situates spheres at each dot and then varies the radii, including going negative in one place).

  • TheGunheart

    This is a $100 game controller that people are just hacking for fun. Just imagine when they start making real cameras based around this tech.

    • Oluseyi

      Actually, the Kinect is the down-market variant of the enterprise/military application of this technology. Look up range imaging for more information.

  • GW

    I know of one possible animation application. A stereoscopic drawing program like SANDDE could be brought to the masses. That and Rhonda’s wireframe could be brought to laymen. Being able to draw in 3D or draw out wireframes from your fingertips is a good start for a system like this, if you ask me.

    http://www.youtube.com/watch?v=SaJUJGpmJGI
    http://rhondaforever.com/

  • http://www.classicparamountcartoons.blogspot.com ParamountCartoons

    Jolly Green giant meets Veggietales.

  • http://scuzzbopper.blogspot.com Ken Priebe

    Crazy! Here is another hacked Kinect piece with a digital bird puppet which a friend sent me.

    http://www.youtube.com/watch?v=tAGnSrdOfyA