Portable Visionbased HCI A Hand Mouse System on

Portable. Vision-based HCI A Hand Mouse System on Portable Devices 連矩鋒(Burt C. F. Lien) Computer Science and Information Engineering Department National Taiwan University

Problems • A Portable Vision-based HCI – Hand mouse operating on a projected interface – Real-time detection of user hand motion from a user PDA/Smart. Phone’s video camera (target platform) • A part of “m. Teller” project – Natural storytelling support system

Why important • Vision-based HCI is a more instinct way to manipulate data • Mobility – Steerable interface everywhere

Related Works I • A Portable System for Anywhere Interactions – Sukaviriya et al. , IBM Research • Real-time hand tracking using a set of cooperative classifiers based on Haar-like features – Barczak 1 et al. , Institute of Information & Mathematical Sciences Massey University

Everywhere Display (IBM) Figure 1: Interactive store application

Related Works II • Real-Time Hand-Arm Motion Analysis using a single Video Camera – Hienz et al. • MMX-Accelerated Real-Time Hand Tracking System – Liu et al. (2001)

Related Works III • Rapid Object Detection Using a Boosted Cascade of Simple Features. – Viola, P. , & Jones, M. (2001). • Robust real-time object detection. – Viola, P. , & Jones, M. • Robust real-time face detection – P. Viola and M. Jones. • Adaboost-based real-time pedestrian detection – P. Viola, M. Jones, and D. Snow.

Target Devices

System Configuration Wireless projector data transmission Hand motion capture and interpretation Interactive Interface

What is new • A portable vision-based HCI on handheld device • Design for general data manipulation

System Implementation • Platform (prototype) – “Laptop” + “Low Cost Camera (USB)” • Software tools – “MS VC++” + “Intel Open. CV library”

Technical Challenges and approaches • Real-time hand motion tracking under an intense lighting condition (projector light) – Adaboost • Hand position vs. Corresponding programs on host – Comparison between PDA’s screen and the projected screen(? ? ) • An efficient algorithm to run detection system on handheld devices (limited computation) – Lower framing rate (2 fps? ) + fewer classifier • Noise filter – A sliding window or a simple filter to filter misbehaving hand motion

Expect Result • A generic vision-based HCI software program running on a laptop (1 st phase) • A user can manipulate the laptop directly on the projected image – Hand mouse (one-click function only)

The End