Virtual Dart An Augmented Reality Game on Mobile

Virtual Dart: An Augmented Reality Game on Mobile Device Supervisor: Professor Michael R. Lyu Prepared by: Lai Chung Sum Siu Ho Tung

Outline Background Information l Motivation l Objective l Methods l Results l Future Work l. Q&A l

What is Augmented Reality (AR)? A combination of real world and computer generated data l Add computer graphic into video l

Background Information Most mobile phones equipped with cameras l Games written in J 2 ME & proprietary development platform l

Background Information l Typical mobile games

Background Information l Mobile games employed Augmented Reality

Motivation l How can the game “remember” external environment? Save external environment information

Objectives l Demonstrate how a game “remember” its external environment for Augmented Reality (AR) l Virtual Dart is just a game for demonstration of the proposed methodology

Problems to be solved… 1. 2. 3. What information should we store? How does the game recognize the information? How does the game perform motion tracking?

Introduction to Mobile Video Object Tracking Engine (m. VOTE) l Convert the camera movement into translational movement and degree of rotation Feature Selection (Find a feature to trace) ü Motion Tracking of Translational Movement ü Motion Tracking of Rotational Movement ü

What is a feature? Section of an image that is easily highlighted for the purpose of detection and tracking l Have a high contrast in relation to its immediate surroundings l X

What does our program need? Functions needed for our program Feature Selection (What information should we store? ) Feature Recognition (How does the game recognize the information? ) Motion Tracking of Translational Movement (How does the game perform motion tracking? ) Can m. VOTE do this? ü û û ü

Program Flow – Initial Algorithm

Experiment of Feature Selection l Feature Selection in m. VOTE VS FAST Corner Detection Algorithm l Testing Environment 1. 2. Normal lighting Insufficient lighting

Normal Lighting Condition Feature Selection in m. VOTE FAST Corner Detector

Insufficient Lighting Condition Feature Selection in m. VOTE FAST Corner Detector

Analysis l Normal Lighting Both algorithms worked reasonably well l Insufficient Lighting Only m. VOTE’s Feature Selection could produce output l l Occasionally, Feature Selection in m. VOTE selected some flat regions as features FAST Corner worked better in terms of accuracy

Experiment of Initial Approach Selected Features Recognized Features

Initial Feature Recognition Conclusion l Accuracy? ? LOW! Selected Features Recognized Features

Program Flow – Enhanced Feature Recognition Algorithm

Enhanced Feature Recognition Set 1 Set 2

Enhanced Feature Recognition Set 3 Set 4

Enhanced Feature Recognition Set 5 Set 6

Algorithms Comparison l Initial Feature Recognition VS Enhanced Feature Recognition l Initial Approach: 3 Features l New Approach: Whole selection area l Reason for LOW accuracy: (Initial Approach) Features may not be descriptive enough

Improvement of Feature Selection l Two conditions of a “Good” Feature: l Descriptive l Large l internal intensity difference Corner Detector can help us to find out good features

FAST Corner Detector Examine a small patch of image l Considering the Bresenham Circle of radius r around the candidate pixel which is called p l Intensities of n continuous pixels on the circle are larger than p or smaller than p by barrier l Potential corner

e. g. r = 3, n = 12, barrier = 25 215 – 65 = 150 > 25 = barrier Marked by red 65 – 39 = 26 > 25 = barrier Marked by Blue 215 255 200 39 167 20 132 16 65 152 91 153 101 165 114 182 167

e. g. r = 3, n = 12, barrier = 25 215 63 21 84 57 108 12 65 90 52 31 15 35 43 57 58 62

e. g. r = 3, n = 12, barrier = 25 215 63 21 84 57 108 12 65 90 90 31 15 35 43 57 58 62

FAST Corner Detector l The typical values of r and n are 3 and 12 respectively l For the value of barrier, we did an experiment to choose the value l We chose “ 25” after the experiment (for what? )

FAST Corner Detector l Advantage: l Fast l Disadvantages: l Cannot work well in noisy environment l Accuracy depends on parameter – barrier

FAST Corner Detector barrier = 10 barrier = 40

How does Feature Recognition works? Full screen as search window l Use Sum Square Difference (SSD) to calculate the similarity of blocks l Still slow in current stage (~20 – 60 sec) l Tried to use a smaller image and scale up to full screen l l Scaling step is too time consuming

Motion Tracking during the game Keep track of three features l Use two features to locate dart board l The last feature point is used for backup l Use if either one of the feature points fail l Condition for a feature point failure l l Feature point is at the edge of the screen l Two feature points are too close

Future Works Allow users to load saved features l Increase the speed of feature recognition l Add physical calculation engine l

Q&A