Hacking XBox Kinect

Hacking XBox Kinect

Merely days after the release of the Microsoft XBox Kinect motion capture accessory, lots of people have started to hack away at it. The immediate deconstructuon and reappropriation of the Kinect has already gained notable media attention, including coverage on the NY Times. Unfortunately, Microsoft has declined to support this innovation.

Kyle McDonald Kinect Pointcloud
Kyle McDonald's XBox Kinect Pointcloud

Among the top hackers, Adafruit industries has put up a bounty for the top hacks, as a way to stimulate the community. So far, a bunch of amazing hacks have come out, with people figuring out how to use the visible light cameras, IR depth cameras, microphones, and servo motors to move the sensor head. Here are a few of the notable ones:

  • Oliver Kreylos combines visual and range data to form a live 3d reconstruction with Kinect

    “By combining the color and the depth image captured by the Microsoft Kinect, one can project the color image back out into space and create a “holographic” representation of the persons or objects that were captured.”

  • Memo Akten draws in 3d and manipulates it by hand – pre-minority report style

    One hand draws in 3D, two hands rotates the view. Very rough prototype.

  • Kinect’s IR Point Projector demonstrated by looking through an IR sensitive video camera (Sony nightshot capable camcorder). Interesting that the Kinect actually projects out its own grid of points to assist in tracking and distance recognition.
  • Early Image Recognition using Kinect, done by Daniel Shiffman. Notice that in this early demo, the system seems to be only using the b/w outline of a given object in order to recognize it. Going forward, by being able to combine b/w edge info, full stereo color camera info, and IR depth data, especially with its own control grid being projected out, object recognition could get very very good. It’s amazing to see the system already so well trained, days after the hardware was released. Daniel also just released the Kinect processing library, which you can download from his blog.