Of all the touch and movement interfaces on the market it looks as though Microsoft may have the simplest solution with Project Natal. It does not require you hold any device like you have to with the Wii and you don’t cover the screen to register input as you have to with touchscreens. The boffins in Ishikawa Komuro Laboratory located at Tokyo University seem to have realized this and come up with their own advanced in-air interface, but for portable devices.
The new technology doesn’t seem to have an official name yet other than being called a vision-based Input Interface. What it allows for is five-finger gesture input on a device like a phone just by using the embedded single-lens camera. Not only that, but it is able to register depth as well as standard 2D movement therefore making it capable of detecting 3D motion input.
Such capabilities allow for a number of advantages over a touchscreen interface. No longer are you covering the screen to register an input,multi-touch is now in-air gestures, and 3D movement registration allows for gestures such as in-air screen tapping as seen in the video above. Because the interface can register such movement typing as if using a keyboard becomes a simple matter of positioning your fingers and tapping in mid-air.
Such an interface will not work with existing devices, however, unless they are capable of capturing 154 fps through the camera and have a forward-facing camera. Depending on how accurate the interface is it could be seen in future mobile devices as either a replacement for or compliment to a touchscreen input method.
This could offer a huge improvement over current touchscreen interfaces if it proves accurate enough and the video above looks promising based on how fast and accurately typing was done using the on-screen keyboard. The ability for the interface to register depth also allows for manipulation of 3D content as well as simple zooming.
What remains to be seen is how much of a battery drain running this high speed camera at 154 fps is. It is surely going to have an impact, but if it’s small then it remains acceptable to install such a camera on a mobile device. We also don’t know how much such a camera costs compared to those commonly used today in mobile devices.
My greatest annoyance when using a touchscreen interface is covering the screen to register input. This in-air system seems to solve that problem without losing any functionality or accuracy. Let’s hope everything else falls into place and we see this system being used in a real product soon.