Immersive Technologies Virtual Augmented and Mixed Reality in

Immersive Technologies Virtual, Augmented, and Mixed Reality in Medicine and Education A VCU Ti. ME Presentation Peter Haar, M. D. Ross A. Yaple M. D.

Immersive Technology: What Is It? “Technology that blurs the line between the physical world and digital or simulated worlds”: ● ● ● Augmented Reality: ○ Real world with an overlay of computer generated content ○ Immersion in an entirely simulated environment ○ Blending real and simulated worlds in ways that they can interact Virtual Reality: Mixed Reality:

How Does It Work? The technology uses the brain’s internal GPS system to convince the viewer that the virtual is real: ● ● ● Visuals: ○ ○ Stereoscopic Displays Frames Per Second (FPS), Field of View (FOV), and Latency Speed ○ ○ Accelerometers, Gyroscopes and Magnetometers Object Recognition Software, Physics of Object Motion, and Interactivity ○ ○ Spatial Audio (3 D) Speech Recognition Motion: Sound:

Who’s Doing It? ● ● Microsoft Hololens: ○ Holo. Anatomy app, Texas Tech, SDSU School of Nursing ○ Children’s Hospital of LA, Stanford Cardiology, Lucie Packard Children’s Hospital ○ USC - Bravemind - exposure therapy ○ Mindmaze (biofeedback), VRHealth, Virtually. Better, Mimerse (therapy applications) Oculus Rift VR: U. S. Military: Private Industry:

Why Is It Significant? VR has the capacity to change the scope of many aspects of medical education: ● Immersion: ○ ● Interaction: ○ ● Learners can be fully involved in increasingly realistic environments and situations, exposing them to real time problem solving, simulation. The breadth of opportunity is as wide as the internet can allow, connectivity within the immersive environment can increase collaboration, empathy. Access: ○ As applications improve and increase, access can improve and increase, exposing more and more learners to the benefits of the technology.

Are There Downsides? ● Immersion “Blurring the Lines of Reality”: ○ ● Cost: ○ ● High fidelity immersion could lead to virtual traumatization of new learners, need to consider the dosing of reality in educational settings. Can be prohibitive, expensive start up costs compared to well researched data on outcomes. Accessibility Standards: ○ Most of this technology is “developing” and has yet to meet disability accessibility standards.

Where Is It Going? As the technology develops, costs will reduce and accessibility will increase: ● Technology: ○ ● Educational Application: ○ ● Use of 360 degree cameras, professional grade digital animation and editing tools can create any type of immersive environment. Use of holographic imaging for 3 D rendering for anatomy, virtual walkthroughs for physiology, biochemistry, pharmacology, etc. Clinical Application: ○ Hi-fi simulations of clinical situations, observations of surgical procedures, virtual therapy groups and modalities, scientific modeling.

What Are The Implications For Teaching and Learning? Opportunities are limited by the imagination: ● ● Solutions at the Level of the Problem: ○ From universal to molecular/atomic, immersion can be at any level. ○ High stakes/low stakes, safety and trust. ○ Will need data to determine how and in what circumstances VR is beneficial. ○ Cost, ethical and accessibility standards must be met. Safety for the Learners: Research Outcomes: Ease of Access:

Applications: Psychiatry ● ● Psychiatry and the Brain: ○ Walkthrough of brain pathways, the synapse and neuromodulation, pharmacology ○ CBT/DBT, Family Therapy, Play Therapy, skills based approaches and observation ○ Scenarios for discussing involuntary hospitalization, dealing with angry parents/family, etc. Teaching Psychotherapy Modalities: Difficult Patient Encounters: Board Certification: ○ Holographic standardized patients for board exams

Applications: Radiology ● Enhanced Visualization of Medical Images: ○ More intuitive spatial visualization of information-rich 3 D datasets Cardiology, virtual colonoscopy, oncology ○ procedures Higher exposure to rare procedures ○ ● ● Simulation of Image-guided procedures: ○ Reduce the risk of inexperienced trainees learning critical Surgical Planning: ○ ○ ● Adding CT or MR images over a patient in real-time VR simulations can reduce surgical planning time and surgical accuracy Anatomy Education: ○ ○ ○ Rich visualization of 3 D relationships Interactive, animated simulations that demonstrate physiology Improved accessibility, will eventually cost less than cadavers

References Educause Learning Initiative. (2017, October 3). 7 things you should know about AR/VR/MR. (ELI Report No. 7149). Retrieved from https: //library. educause. edu/resources/2017/10/7 -thingsyou-should-know-about-ar-vr-mr. Craig, Emory & Georgieva, Maya. (2017, August 30). VR and AR: driving a revolution in medical education & patient care. Educause Review. Retrieved from https: //er. educause. edu/blogs/2017/8/vr-and-ar-driving-a-revolution-in-medical-education-andpatient-care.