Virtual Environments Introduction Anthony Steed Simon Julier Department

Virtual Environments: Introduction Anthony Steed, Simon Julier Department of Computer Science University College London http: //www. cs. ucl. ac. uk/teaching/VE

Outline • What are Virtual Environments? – Working definition – Constituents of the environment • Technologies for VEs – What are the toolbox of parts? • Describing and Measuring a VE – Immersiveness – Levels of virtualisation – Presence

Defining Virtual Environments • What are Virtual Environments? – Working definition – Constituents of the environment • Technologies for VEs – What are the toolbox of parts? • Describing and Measuring a VE – Immersiveness – Levels of virtualisation – Presence

Virtual Reality in the Early 90 s NASA Ames Photo Archive

Device-Driven Definitions of Virtual Environments • • • Virtual Reality is electronic simulations of environments experienced via head mounted eye goggles and wired clothing enabling the end user to interact in realistic three-dimensional situations. (Coates, 1992) Virtual Reality is an alternate world filled with computer-generated images that respond to human movements. These simulated environments are usually visited with the aid of an expensive data suit which features stereophonic video goggles and fiber-optic data gloves. (Greenbaum, 1992) The terms virtual worlds, virtual cockpits, and virtual workstations were used to describe specific projects. . In 1989, Jaron Lanier, CEO of VPL, coined the term virtual reality to bring all of the virtual projects under a single rubric. The term therefore typically refers to three-dimensional realities implemented with stereo viewing goggles and reality gloves. (Krueger, 1991, p. xiii) “Defining Virtual Reality: Dimensions Determining Telepresence”, Steueur, 1992

Device-Driven Definitions of Virtual Environments User Synthetic Environment User Computer Interface Devices This is what makes this a VE Real Environment

Limitations of Device-Driven Definition • “VR” and “not-VR” systems are defined by presence / absence of hardware alone – Previous definitions highly image oriented • Lack of theoretical dimensions for parameterising characteristics of VR – How can different VR systems be compared? – How can VR be compared with different media?

A VE is an Immersive, Mediated Communication Medium User Synthetic Environment User Environment Interface Devices Mediated Medium Real Environment

Components of the Environment Contents: Actors and Objects Geometry: Dimensions, Metrics and Extent Dynamics: Interaction Rules “Virtual Environments and Environmental Instruments”, S. Ellis, 1996 Environment

Contents • Environment is made up of discrete items known as objects and actors • Objects – Discrete and identifiable – Described by property vectors • Actors are objects that initiate interactions • The self is a special kind of actor with a point-ofview

Geometry • Description of the environmental field of action • Contains: – Dimensionality: The degree of freedom of the position vector – Metric: The basic mathematical rules for defining order, distance, etc. – Extent: The range of possible values of the position vector

Dynamics • These are the rules of interaction among the contents • These can be: – Differential equations of Newtonian dynamics to describe kinematic and dynamic relationships – Grammatical rules for pattern-matched triggered actions • For comfortable use, relationship between proprioceptive information and sensory data requires consistency, predictability, and completeness

VE Technologies • What are Virtual Environments? – Working definition – Constituents of the environment • Technologies for VEs – What are the toolbox of parts? • Describing and Measuring a VE – Immersiveness – Levels of virtualisation – Presence

Components of a VE System • • Display / Rendering System User Input Tracking Database - no time to talk about this

Visual Display Systems Headmounted Display Fish. Tank. VR: VR on a Monitor Headmounted Display

CAVE = Cave Automatic Virtual Environment Cruz-Neira, C. , Sandin, D. J. , De. Fanti, T. A. (1993) Surround-Screen Projection. Based Virtual Reality: The Design and Implementation of the CAVE, Computer Graphics (SIGGRAPH) Proceedings, Annual Conference Series, 135 -142.

Audio Display Systems Headphones Spatialised Audio

Haptic Display Systems Cyber. Grasp Phantom Tactors

Olfactory Display Systems “Projection-Based Olfactory Display with Nose Tracking” Y. Yanagida, S. Kawato, H. Noma, A. , and N. Tetsutani, IEEE VR 2004, Chicago

Galvanic Vestibular Response System

User Input Devices Cubic Mouse Wand Cyber. Glove

Tracking Systems Inside-Out hybrid camera inertial system Outside-In camerabased tracking system

Describing and Measuring VEs • What are Virtual Environments? – Working definition – Constituents of the environment • Technologies for VEs – What are the toolbox of parts? • Describing and Measuring a VE – Immersiveness – Levels of virtualisation – Presence

Why Study VEs? • VEs are tools to try to maximise information between the computer and user • Input – Responds to the user through 3 D controls – Possibly speech controlled • Output – High-fidelity surrounding graphics – High quality spatialised sound • The input / output configuration needs to maximise the information flow

Optimising the Performance of a VE • “I define a virtual reality experience as any in which the user is effectively immersed in a responsive virtual world. This implies user dynamic control of viewpoint. ” - Professor Fred Brooks, UNC Chapel Hill, USA • But what does “effectively immersed” mean?

Optimising the Performance of a VE • The best performance occurs when: – the user loses awareness of the real world props that support the perceptual illusions; and – Actions are successfully supported in the environment • In other words, the user experiences presence • Presence is achieved in several steps: Immersion Virtualisation Presence

Immersion • Immersion is a description of the technology – – – – Inclusive – sensory experience from VE only Extensive – more sensory modalities Surrounding – from all directions Vivid – high fidelity Egocentric – first person point of view Plot – things are happening and the VE responds to the user Proprioceptive match – between sensory data and proprioception

Virtualisation • Immersion supports the process of virtualisation • Virtualisation – “the process by which a viewer interprets patterned sensory impressions to represent objects in an environment other than that from which the impressions originate”.

Levels of Virtualisation • Virtual Space – see a 3 D object when looking at something inscribed on a flat sheet – Perspective cues, lighting, shading • Virtual Image – perception of an object with depth – Stereo disparity • Virtual Environment – perception of a surrounding environment – head-slaved parallax, accommodation and vergence, surrounding display

Virtual Space Half-Life 2, Valve Software

Virtual Image

Virtual Environment • Requires (almost) full immersion because of the requirements for headrelated and egocentric display • At this point, user can actively explore the (visual) environment by moving naturally

Some Presence Definitions • ‘The sense of “being there”’ (Held & Durlach, Sheridan, Zeltzer: premier issue of PRESENCE, 1992) • ‘A perceptual illusion of nonmediation’ (Lombard and Ditton, 1997) • ‘A mental state in which a user feels physically present within the computer-mediated environment’ (Draper & Kaber, 1998) • ‘The subjective experience of being in one place or environment, even when one is physically situated in another’ (Witmer & Singer, 1998)

Presence Operationally • Successful substitution of real sense data by computer generated sense data • ‘Successful’ – response is similar to expected response in everyday reality • ‘Response’ – – Low level physiological high level cognitive and emotional – Includes verbal responses about ‘being there’ – Response includes potential for interaction

Impact of Virtual Environments • Meehan, M. , B. Insko, M. Whitton and F. P. Brooks Jr. (2002). "Physiological Measures of Presence in Stressful Virtual Environments. " ACM Transactions on Graphics 21(3): 645 -652. (Proceedings of ACM SIGGRAPH 2002, San Antonio , Texas ).

Summary • A working definition of VEs – Mediated communication environments • A very brief overview of the technology behind virtual environments • Levels of virtualisation • Critical role of immersion in affording proprioceptively correct changes in sensors data • Presence as the human response