LowCost MultiTouch Sensing through Frustrated Total Internal Reflection

Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection Jefferson Y. Han, New York University Presented by: Cody Boisclair

Introduction l l l Touch sensitivity is fairly common in electronics today… …but only for a single point of contact at any given time. Touch-sensitive devices that allow for multiple points of contact have thus far been expensive or difficult to produce.

Why multi-touch? l Allows a user to interact with a system with more than one finger at once l l l Chording Operations using both hands Also allows multiple users to interact on the same touch-sensitive platform l l Interactive walls Interactive tabletops

Prior approaches to the problem l Matrix of smaller sensors l l l Requires many connections, which severely limits resolution Visually opaque - no touch-screen Video cameras l l l Measuring of brightness Stereo cameras Identifying tracking markers in gel

A new approach… F rustrated T otal I nternal R eflection

Total Internal Reflection l l When light encounters a medium with a lower index of refraction (e. g. , going from glass to air), its refraction depends on the angle at which it hits the border. Beyond a certain critical angle, light is not refracted, but instead reflects entirely within the material. l This is the basis for fiber optics and other optical wave guides.

Frustrated Total Internal Reflection l l If another material touches that within which the light is reflecting, the reflection is frustrated, causing the light to escape. This has been used in the past: l l l fingerprint imaging early touch sensors (1970 s!) tactile sensors for robotic grips

A Schematic of FTIR (Kasday, 1984)

Using FTIR for touch sensitivity l A clear acrylic sheet is used as the touch surface. l l 16 inches x 12 inches in prototype Edges of surface lit by infra-red LEDs to produce total internal reflection. A video camera is mounted under the surface and facing it. When the surface is touched, the light escapes and registers on the camera.

Using FTIR for touch sensitivity l l l Basic image-processing techniques are performed on the camera output to identify the points of contact. Computer-vision techniques are used to interpret the motion of contact points as discrete touches or strokes. Processing easily handled in real-time by a 2 GHz Pentium IV processor

Advantages to this approach l High capture rate and resolution l l l 30 frames per second 640 x 480 True zero-force touch sensitivity Inexpensive to construct Scalable to much larger (even wallsized!) surfaces Transparent: can be combined with rearprojection display

How is projection achieved?

Disadvantages l l Requires significant space behind touch surface for camera Gloves, certain types of styluses, and even dry skin may not register l l a function of refractive index Residues on surface (e. g. , sweat) also produce FTIR effect that may build up

Some applications… Video Clip at NYU (Local Copy) l l Uses a 36 x 27 -inch rear-projection screen. Touch information is sent to programs using OSC (Open Sound Control) protocol

Any Questions?