String SDK

Geek Out Monday. 3D Video on an iPad via Kinect.

It’s Monday, so I thought I would start the week with a geek fest featuring some 3D video built with a Microsoft Kinect, and played back on an iPad.

LAAN Labs, used String Augmented Reality SDK to display the video and audio that was recorded with the Kinect. Working with Libfreenect’s open Kinect project, they recorded the incoming data from the Kinect, and then built a textured mesh of the subject from calibrated rgb and depth data sets. This was done for each frame in the sequence which allowed the video to be played back in real-time. Using a simple depth cut off, they were able to isolate the person in the video from walls and other objects in the room.

The image was projected onto a printed image marker in the real world using the String SDK. That image was then used as a QR marker for the iPad to read and display the image.

While this is pretty rough, the result is still impressive, and it really shows off the power of Kinect’s open source community, String SDK, and the Open Kinect Project. I can’t wait to see how this develops. The potential for content development here is huge.