2009 has seen Click Suite knee-deep gestural interfaces and multi-touch technologies. While our recent project for the Auckland Museum and their beautiful Wonderland exhibition allowed us to deliver this on a large scale (separate blog and video to follow shortly!), we've also been dabbling in smaller, more DIY, approaches.
Following these instructions we were amazed with what you can achieve simply using a photo frame, webcam, cardboard box and an A4 piece of tracing paper (gaffer not included). We were also pleasantly surprised how quick it was to get up and running.
We translated the data from the camera using Community Core Vision, described as “an open source/cross-platform solution for computer vision and machine sensing”.
This takes a video input stream and then uses blob-detection to output tracking data (e.g. coordinates) and events (e.g. finger down, moved and released). The hand gestures were handled using the Microsoft Windows 7 built-in multi-touch library.
Initially this set-up provided us with a fairly good tracking performance but we found that once we took it into an actual software application the camera's frame rate degraded and started to interfere with the performance of the blob-detection.
Progen Labs used similar techniques for this 80" Wall for LOUIS XIII at the China (Shanghai) International Boat Show 2009.
LOUIS XIII 80" Wall from progen on Vimeo
The video is quite long so once you get a look at the setup, I recommend skipping through to around 4:45 for a look at the actual screen in action. They also had to tweak the calibration of the blob-detection to minimize jitter, but it’s a great example of how the simple idea of the cardboard box can be applied in the wild.
Our next steps have been to take the raw video data straight from the camera into Processing using open-source code to provide the blob detection. This means it's a more direct path from the camera to the application and we are having positive results with performance so far.
It's has been very exciting to see and be part of developing new ways of interaction beyond the point-and-click of the mouse. As we saw from the way people engaged with our SuiteSpot experience, gestural interfaces provide a more immersive interactive experience for the user. Now we’re looking forward to getting stuck into the wonders of multiple inputs and the endless possibilities that come along with it.
This post has been a joint effort between myself and fellow multi-touch enthusiast, Adam Harte.
