Butterfly Effect An Augmented Reality Puzzle Game Marleigh

Butterfly Effect: An Augmented Reality Puzzle Game Marleigh Norton Blair Mac. Intyre Steven Dow Maribeth Gandy

Game Overview • Spatial puzzle where physical world is not modeled • Virtual butterflies in physical environment • Catch them! – Butterflies are stationary – Challenge comes from reaching the butterflies • Bring butterflies into reach by rotating the virtual world – Virtual world rotates in 90 degree chunks about a player defined axis.

Equipment

Head-Mounted Display

Tornado Stick

Rotations • Tornado Stick controls Virtual Axis – First Button – Place / Remove Axis – Second Button – Pause / Unpause Axis – Third Button – Rotate Butterflies

Tornado Stick • People had trouble understanding rotations – Not everyone is used to 3 D geometry – Though 3 D puzzles have worked before • Change the metaphor, not the interaction – “Tornado Stick”

Butterfly Effect

Butterfly Effect Video

Design Context • Start with medium, then design game – Mapping physical and virtual worlds is hard – AR works best when user is moving slowly • Tracking more accurate • Safer for player – Cognitive Tunneling

Design Context • Home environment – Run on next generation game consoles – Must be easy to set up – Minimize use of special equipment

1 st Prototype: 2 D Paper

2 nd Prototype: 2 D Interactive

3 rd Prototype: Video

Tuning • Two Major Challenges – 90 degree rotation constraint – Understanding virtual space and physical space • Possible Changes – Loosening rotation constraint – Adding more manipulations (translations) – Adding better depth perception cues

Open Issues: Depth Perception • Static Depth Cues

Open Issues: Depth Perception • Remove Automatic Depth Cues

Open Issues: Depth Perception • Remove Unavailable Depth Cues

Open Issues: Depth Perception • Texture probably the best

Without Grid

With Grid

Is Occlusion Really Impossible? • Ad hoc modeling – Automatic modeling during calibration – Fiducials • Real time depth – Depth from stereo • Something to be explored

Virtual Obstacles - Bees • No consequences for using poor strategy • Bees behave like butterflies • “Sting” if caught • Penalty – Lose life – Butterflies escape

Questions?

Old Slides from Past Talks (In case people as a question I have slides for. )

Butterfly Effect: An Augmented Reality Puzzle Game Marleigh Norton Technical Advisor: Blair Mac. Intyre

Person Walks Through Game Area

Person Walks Through Game Area

Person Walks Through Game Area

Derive Walls from Path

Derive Walls from Path

Why this is Fun • Fun in the same ways Rubik’s Cubes are fun • Caillois’s Four Categories of Games [1] – agôn, alea, mimicry, ilinx • Mac. Tavish’s “astonishment at visual and auditory technology” [2]

Open Issues • Scoring • Capture Interaction • Depth Perception

Open Issues: Scoring • Score based on efficiency, not speed • Penalized for moving too much – Accrue score as you walk – High score bad • Problem: High Score Traditionally Good

Open Issues: Capture Interaction • Location-Based Capture – Capture butterfly by getting close enough – Pros: Technically simple. No additional cost. – Cons: Precise head tracking more important. Less thematically appropriate. Less fun. • Butterfly Net Capture – Capture butterfly in a net – Pros: More active, thematically appropriate interaction. – Cons: Requires an additional tracking system, likely expensive.

Open Issues: Depth Perception • Judging depth in virtual worlds is difficult – Overcoming this difficulty is part of the game, yet it should not be impossible • Problem: What can we do to help the player understand the relative locations of the butterflies? • Research on Depth Perception yields several techniques [4] [5] [6] [7]

Questions & Feedback • Scoring – Efficiency • Capture Interaction – Location-based capture – Butterfly Net Capture • Depth Perception – Which cues to use? More ideas? • Other Feedback

References: Game Theory • [1] Caillois, Roger, “Man, Play, and Games, ” trans. Meyer Barash, University of Illiniois Press, 1961 (orig. 1958) ch 2. • [2] Mactavish, Andrew, “Technological Pleasure: The Performance and Narrative of Technology in Half-Life and other High-Tech Computer Games”, in: Geoff King and Tanja Krzywinska (eds. ) Screenplay: Cinema/ Videogames/ Interfaces (London: Wallflower Press, 2002) 33 -50

References: Depth Perception 1 • [4] Cutting, J. E. , and P. M. Vishton, 1995. “Perceiving layout and knowing distance: The integration, relative potency and contextual use of different information about depth”, in Perception of Space and Motion, W. Epstein and S. Rogers, Eds. Academic Press, New York, 69 --117. • [5] Furmanski, Chris, Ronald Azuma, and Mike Daily. “Augmented-reality visualizations guided by cognition: Perceptual heuristics for combining visible and obscured information”, in Proc. IEEE and ACM Int'l Symp. on Mixed and Augmented Reality (ISMAR 2002) (Darmstadt, Germany, 30 Sept. - 1 Oct. 2002), pp. 215 -224.

References: Depth Perception 2 • [6] Sinai, M. J. , W. K. Krebs, R. P. Darken, J. H. Rowland, and J. S. Mc. Carley, 1999. “Egocentric distance perception in a virtual environment using a perceptual matching task”, in Proceedings of the 43 rd Annual Meeting Human Factors and Ergonomics Society, 43, 1256 -1260. • [7] Surdick, R. T. , E. T. Davis, R. A. King, and L. F. Hodges, 1997. “The perception of distance in simulated visual displays: A comparison of the effectiveness and accuracy of multiple depth cues across viewing distances”, in Presence: Teleoperators and Virtual Environments 6, 5 (October), 513 --531.

References: DART • [8] Mac. Intyre, Blair, Maribeth Gandy, Steven Dow, and Jay David Bolter. "DART: A Toolkit for Rapid Design Exploration of Augmented Reality Experiences. " To appear at conference on User Interface Software and Technology (UIST'04), October 24 -27, 2004, Sante Fe, New Mexico.

Implementation Status • It’s playable!* – Populate the world with a pre-defined set of butterflies – Butterflies can be captured – Tallies of caught and remaining butterflies are displayed – Butterflies can be rotated about the axis – Physical axis interface * When all of the equipment is working.

The Screen

Feedback • • Rotation Interface Depth Perception Score Aesthetics

Rotation Interface • Problem: Rotating about an axis? Huh? – Suggested alternatives: • Randomization • Jar, Mini-version of gamespace – (Rotation about a point) • “Tornado Stick” • Changes: Tornado Stick – Axis/Tornado Stick only interface in which all manipulations are valid moves – Arrows pointing to butterflies’ final location

Depth Perception • Problem: It’s hard to tell where the butterflies are, especially when they’re “behind walls” – Complication: can’t change appearance of butterfly based on presence of wall (or can we…? ) • Changes: Add grid with drop shadows – Also, it’s less of a problem when you’re actually playing

Score • Problem: for Efficiency-based score, lower was better (confusing players) – Suggestions • Timer • Subtract from High initial score • No Score • Changes: No Score – Score isn’t really that important – Was causing more problems than it solved

Aesthetics • Problem: What will the game look like? – Collecting the butterflies should be compelling • Changes: Aesthetics more fully fleshed out – Childlike graphic design with scanned crayon textures – Player is frog, eating butterflies

Frog has Butterfly in Sights

Frog Catches Butterfly

Frog Eats Butterfly

Future Work: Finish Implementation • • Sounds Graphics Animations Refine Stick – Are those the right buttons? – Buttons or Switches or ? ? ? – Inertia. Cube Issues with Ferris Environments

Thanks to the Augmented Environments Lab, especially: • Blair Mac. Intyre • Steven Dow • Enylton Machado Coelho Questions?