Lecture 1 Introduction to Virtual Reality Virtual Reality

Lecture 1. Introduction to Virtual Reality 고려대학교 그래픽스 연구실 Virtual Reality

Contents • • • What is VR? Conceptual Model of VR VR Related Areas History State of the Art and R&D Issues Some Application Examples

Science vs. Engineering • What is not VR. • high road - replication of reality low road - 3 -D interface / interaction • VR = studies on reality • analogous to AI – lots of hypes – AI as a science / engineering computational reality

What is not VR.

• VR as computational reality – To seek for the computational model of reality. – To apply the model to the VR system. • VR can be understood in the context of modeling efforts. – – – intelligence linguistics emotion life reality compuational “X”

Definition : What is VR? • Computer generated environment that is … – Immersive (like IMAX, dome projection), – Interactive (like computer game), – Multi-sensory, – Viewer-centered, – 3 -D, and – The combination of technologies required to build such environments.

VR as a computer technology • Technological trend – powerful – smart – physical • computer - human interaction • primary concern --- software other important issues --- hardware, human factors, social issues, infrastructure

Interacting with computers Computer World

New Paradigm for HCI Conventional Computer World VR-based Computer World

VR as a media • All medium attempt to create the “virtual presence”. – Theater, poem, fine arts, novel, telephone, movies, TV, . . . • VR vs. existing media – – – immersive interactive 3 -D multimodal mediated • Information is not sent back and forth. • Mediated environments are created and then experience.

• effectiveness of communication depends on. . . the sense of “being there”. • virtual presence depends on. . . – sensory breadth – sensory depth – interactivity

VR Application Education/ Training Design Engineering Entertainment Museum Arts VR application Scientific Visualization Medicine Communication Products Engineering

VR technologies Character CRT Computer graphics Graphic CRT Realtime CG Virtual console Keyboard Mouse tablet 3 D mouse Computer simulation/ visualization Virtual Reality Video arts Virtual products design 3 D CAD Computer Aided Design Tele-conference Tele-Existence TV phone Tele-Operation Telephone All technologies meet together at VR !!

VR in Real World vs. Virtual World Tele-Existence in Real World Virtual Reality • Standard Tele-Existence • Augmented Tele-Existence • Size • Sensation • Time • Physical World Tele-Existence in • Quasi Physical World Virtual World • Non Physical World

Conceptual Model of VR Display system • 3 D image Large-scale display, Head Mounted Display • Sound field by DSP • Force beedback mechanism • Tactile display Computer Simulation System Human Sensing system • Non-contact type magnetic field supersonic wave infrared light • Contact type optical fiber strain gauge potentio-meter

VR Related areas 1. Training simulation 2. Tele-operation 3. Computer graphics 4. Artificial intelligence

Training simulation • Differences – – reconfigurable by changing software may include highly unnatural environment highly interactive and adaptive use of a wide varielty of human sensing modalities and sensorimotor systems – highly immersive – near-field is synthetic; far-field is synthetic.

Tele-operation • for at least 30 years. • Tele-operator – directly (manually) controlled tele-operator – tele-robot • Tele-operation vs. Virtual reality • Tele-presence vs. Virtual presence

Computer graphics • • Modeling Motion control (animation) Rendering User interface

Artificial intelligence • Studies on perception and cognition • Testbed for AI research

History • 1’st stage: some visionaries – – Morton Heilig : Experience. Theatre(1962) Ivan Surtherland : Sketchpad(1963), HMD(1966) Myron Krueger : Artificial Reality(1972) William Gibson : “Cyberspace” in Neuromancer(1984) • 2’nd stage: technology development for specific purposes – training simulator : Earlier works – space exploration : NASA for astronaut simulation – tele-operation

History(con’t) • 3’rd stage: VR as the general-purpose technology – Jaron Lanier : VPL(1987) – Dataglove, Eye. Phone, VR system – VR industry : Division Ltd. Sense 8, World. Design(production house, W-Industry(game) – VR academia : MIT, UNC, UW, Tokyo U. • Next stage: Toward a scientific discipline – – – computational reality a new computing paradigm a new media a new art form representation, creation and operation of virtual worlds

State of the Art & Issues • Reference – Virtual Reality: Scientific and Technological Challenges”, pp. 35 -66, National Research Council, National Academic Press, 1995. • Areas of the study – – application domains psychological issues VR technologies evaluation of VR systems

1. Application domains • • • design, manufacturing & marketing medicine, health care hazardous operations training entertainment, military experimental psychology education information visualization tele-communication, tele-travel

2. Psychological topics • human performance characteristics • alteration of sensori-motor loops • developing the cognitive model • cognitive side-effect

3. VR technologies • Gap between the current technology the required technology (exception -- entertainment, tele-operation) (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

Human-machine interface (cont’) • • visual channel auditory channel haptic channel motion interface position tracking video camera microphone others

Visual channel • visual display – HMD – OHD (off-head display) • perceptual effects – – – mis-registration sensori-motor alteration distortion time-delay noise

• research issues – – – – ergonomics improvement of resolution and fov wireless integration of visual, auditory, position tracking sun glass-like see-through option exploiting foveal and peripheral vision

Auditory channel • Current hardware is adequate. • Research issues – perceptual issues • similar to the visual channel • use for sensory substitution (for visual, haptic) – auditory scene analysis – hear-through display

Position tracking and mapping • tracking = finding a point mapping = finding a 3 D surface (e. g. , environmental mapping) • tracking mechanisms – – – mechanical linkage magnetic optical acoustic intertial

• eye tracking • research issues – tracking – mapping

Haptic channel • force, pressure, tactile feedback • unique characteristics – Haptic interface requires manipulation and sensing • mechanism – body-based -- glove, exoskeleton – ground-based -- joystick

• Research issues – haptic science = study on the human haptics (bio-mechanical, psychophysical, cognitive) – tool-hand system (which takes its metaphor from real tools. ) – creating the haptic illusion – the interaction effects of haptic and vision – texture, temperature devices

Motion interface • motion – whole-body motion • passive -- e. g. , motion platform • active -- e. g. , locomotion – part-body motion • passive • active

• motion cues – – – vestibular system -- inertial motor visual auditory proprioceptive / kinesthetic -- muscle tactile

• motion interface – inertial system • moves the body (e. g. , treadmill, motion platform) – non-inertial system • simulates motion

Other types of interfaces • olfactory (smell) • gustatory (taste) • heat, wind, humidity • speech • direct physiological sensing and control

VR technologies (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

(2) Generation of virtual environments Display system • 3 D image Large-scale display, Head Mounted Display • Sound field by DSP • Force beedback mechanism • Tactile display Computer Simulation System Human Sensing system • Non-contact type magnetic field supersonic wave infrared light • Contact type optical fiber strain gauge potentio-meter

Generation of VE(cont’) • the core issue • general-purpose VR system? • trade-off between realism and interactivity • requirements – frame rate – response time – scene quality

Generation of VE(cont’) • hardware • interaction and navigation • VE management – simulation – rendering • • modeling autonomous agent hypermedia interaction OS

VE management - simulation • Task : simulating everyday world • Traditional simulation methods do not work. (requires pre-processing) • Need : “meta-modeling”

VE management rendering • Issue : load balancing – 1. partitioning VE – 2. LOD • Much work has been done on static scene. • Research issues – 1. dynamic scene – 2. parallel rendering

OS • real-time, multi-modal requirements • very high-resolution time slicing • atomic, transparent distribution of tasks • large number of light-weighted processors, communicating by means of shared memory • support for time-critical computing: – negotiated, graceful degradation – guaranteed frame rate, lag time

VR technologies (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

(3) Tele-robotics • tele-robotics and VR • hardware • time-delay problem • distributed tele-robots

(4) Network • The future is here! • applications – – distance learning group entertainment distributed training distributed design • current • future • What is needed

Research Organizations International Efforts U. S. A. medicine U. K. Germany Japan HDTV. defense, space, visualization, education, training, entertainment VR as a logical extension of robotics, automation,

Academia • • • HIT Lab, University of Washington University of North Carolina Media Lab, MIT Georgia Institute of Technology Naval Postgraduate School University of Pennsylvania University of California at Berkeley University of Illinois - Chicago Columbia University of Toronto

And, Some VR Example… (Video)