Visualizing the Brain in 3D
The brain is more challenging to understand than heart, kidney, skin, and other organs1. Unfortunately, its complex interconnecting structures and interactions have no close similarity to common, everyday experiences or models2. Cutting open a brain reveals globs of gray and white with very little of its amazing complexity or vast potential. Medical-related learning, such as the pathways and circuits relevant to addiction, are certainly not obvious or easy to understand.
Students need to develop an internalized, spatially relevant 3D map of brain structures and their connections 3. Poor 3D comprehension can lead to errors, 4 misunderstandings, or doubt that a scientific basis of a brain-based disorder exists. To better understand student needs, we surveyed 36 students and 47% noted that health professional school introduced a new challenge in learning 3D structures, while 50% described it as initially very difficult.
Currently, students use 2D diagrams, dissections5, radiological scans 6,7, textual descriptions8, 3D visualizations represented on a 2D screen9, glossaries 10,11 and slides12–14 to visualize brain neuroanatomy. 3D neuroanatomic renders delivered on a 2D screen can be partially successful15–17in providing spatial relationship abilities 18–20 compared to standard 2D objects1,21,22. But these tools still do not reach the full potential of 3D immersive environments to enhance spatial skills and develop 3D maps. Learners need a powerful tool to visualize the brain and time-efficiently build an internal model of the brain. One does not exist. We asked 36 students about existing educational tools and forty-seven percent (47%) of the respondents to the survey disagreed that existing tools were adequate and 80% agreed that they wished there were better tools to understand 3D structures.
Virtual reality training engages the learner in environments not feasible in the real world, such as immersion inside body systems23 used in surgery training 24.
Older VR solutions25 were bulky and expensive. Prior solutions using head-mounted displays suffered from a lag between head movement and visual display, leading to a lack of presence26[per scales to measure presence27,28] and often to severe motion-sickness side effects.29 Room-sized technologies such as Cave230 and C631 could produce an amazing experience 32–34 at an equally amazing cost.
Oculus Rift™ VR renewed enthusiasm for consumer VR technology.35 Oculus took advantage of Moore’s law alongside improvements in high-resolution small screens utilized in the latest generation of iOS and Android smartphones. For the field of VR, the ability to build a lightweight VR device connected to a computer 36 was revolutionary. The Oculus VR experience compared favorably37 to the room-sized C6 simulator 31, a multi-million dollar room with displays surrounding the user. The 3D stereoscopic VR world of today’s headsets responds promptly to head movements, fills the entire field of vision, and smoothly generates images that create a convincing alternate reality where the user can look up, down, and behind them.
These headsets allow a near-perfect representation of reality in terms of low lag and a full field of view.38 The realism that is possible with the device evokes “real world” responses39 that one simply does not see when folks watch a video or 3D images displayed on a 2D screen.
The technological potential application of immersive virtual reality to medicine is growing rapidly40. Headset VR technology has already been used to explore pain control41, surgical repair of hernia42, and similarly complicated objects for engineering training43.
The NIH BRAIN Initiative and the component Human Connectome Project have highlighted technologies that depict the brain’s gray and white globs in stunning images. Yet, as glorious as these images are, they are still displayed on a 2D screen. Although diffusion tensor MRI and images produced by tractography are an improvement on depictions of the brain based on standard MRI and diagrams based on dissection and other imaging techniques, they are still not immersive.
3D Immersive Learning Environment
In medical education, 3D graphics delivered on a 2D display are more efficient than textbooks 44 and standard 2D objects 1,21,22. However, full immersion in a VR environment can enhance learning even more. Immersion involves a complex interaction between the psychological experience of feeling “present” within the learning model, representational fidelity, and learner interaction.45,46 The immersive learning experience provides advantages over routine education or even 3D models displayed on a 2D screen including superior memory, transfer, and motivation.47–50 Immersive learning allows for a great connection to the materials being presented51 and a sense of presence [per the Temple Presence Inventory.27,28,46].
Spatial memory is essential to survival and in the past helped our species understand things like caves, hunting grounds, and collections of people.17,52 In modern-day, when faced with something new, we quickly and efficiently create internal 3D models simply by looking around and walking through an environment.53–55
Graphical representations (in 3D models delivered on a 2D display) enhance understanding of complex spatial relations associated with neuroanatomy when compared to 2D representations.1,21,56,57 Early work in the spatial memory field explained that construction of spatial memory and cognitive maps depends on consistent input from a variety of information for both rewarded (goal-directed) and non-rewarded (exploratory) purposes and that virtual reality may enhance the effect further58(p459). Virtual environments do indeed confer spatial knowledge of the components and refine the understanding of how these components interact.23 Our immersive 3D experience does not require the spatial manipulation skills inherent in visualizing a 3D object on a 2D display1, and thus is likely to be even more effective than 2D-delivered 3D. The “supernatural” ability to instantaneously transport or zoom in/out of different scales may further enhance spatial learning 59(p279).
Internalized 3D models stored in spatial memory are strong, potentially explained by the uniqueness or novelty60 of the experience. The subsequent spatial map may cause understanding to persist longer than visualization based on images or textual descriptions.
Augmenting reality with labels and guides further improves spatial memory acquisition16,61 and ensures the learner does not become “lost” within the brain. The background, too, is made to help orient the learner regarding their current perspective.
With cognitive abilities engaged, learners utilize the layout and landmark orientation elements of the VR experience to develop a cognitive map that includes the overall layout, the objects, their meaning, and purpose.62,63 Such a map organizes, structures, and stores memories of associations and concepts in durable memories that are more easily retrieved.64,65 The internal map, with its relationships among elements in a space, also translates to a better understanding of relationships among concepts.66,67
To support immersive learning of the brain’s structures, we are creating VR Brain Exploration, an immersive 3D brain simulation that future medical professionals explore via the new generation of virtual reality headsets. Students will have a tool to build an internal 3D representation of the brain that yields improved understanding of the brain’s complex structures and the network of neural pathways compared to standard 2D-delivered training. The improved internalized brain map may, in turn, result in increased interest and confidence in the medical basis of addiction and other brain-based illnesses, as well as the effect of substances on brain circuits and neurotransmitters. An improved map will aid the understanding of new knowledge of the brain as it arises. Our work can also guide the exploration of immersive VR into additional medical topics and target different learning styles.
Our proposed 3D virtual reality VR Brain Exploration confers understanding of the complex neuroanatomy and neurophysiology of the brain and its processes and helps students develop a complete model of the brain. We look forward to developing this immersive VR experience so that learners can go into the brain and get a real understanding of anatomy from the inside out.
- MRI T2 Brain axial image: GNU Free Documentation License
- ClinicalTools / HealthImpactStudio
- DTI Brain Tractographic Image Set per Aaron Filler, MD, PhD, GNU Free Documentation License
- Brewer Danielle N, Wilson Timothy D, Eagleson Roy, de Ribaupierre Sandrine. Evaluation of neuroanatomical training using a 3D visual reality model. Stud Health Technol Inform. 2012;173:85-91. doi:10.3233/978-1-61499-022-2-85.
- Anatomy Warehouse. Classic heart anatomy model. 2014.
- Brennan Troyen A, Leape Lucian L, Laird Nan M, et al. Incidence of Adverse Events and Negligence in Hospitalized Patients. N Engl J Med. February 7, 1991;324(6):370-376. doi:10.1056/NEJM199102073240604.
- Brennan T, Leape L, Laird N, et al. Incidence of adverse events and negligence in hospitalized patients: results
of the Harvard Medical Practice Study I*. Qual Saf Health Care. April 2004;13(2):145-152. doi:10.1136/qshc.2002.003822.
- Macchi Veronica, Porzionato Andrea, Stecco Carla, Parenti Anna, De Caro Raffaele. Clinical Neuroanatomy Module 5 Years’ Experience at the School of Medicine of Padova. Surg Radiol Anat SRA. April 2007;29(3):261-267. doi:10.1007/s00276-007-0201-9.
- Lindsley T. Community Spotlight: Creating a Virtual Brain Atlas for Medical Education using ITK. 2012.
- Johnson Keith A, Becker J Alex. The Whole Brain Atlas. 1999.
- NIDA. Animation: The rise and fall of the cocaine high. Natl Inst Drug Abuse. November 7, 2014.
- Coherent. Coherent. Rx 2014. 2014.
- Oxford University Press. Sylvius 4 Online: An Interactive Atlas and Visual Glossary of Human Neuroanatomy. 2017.
- Martin John H, Soliz Ewa. Interactive Neuroanatomy Atlas. 2003.
- Kumar Vipin. MRI SECTIONAL ANATOMY OF BRAIN. February 2, 2013.
- Kharabish, Mohamed. CT and MRI Interpretation. January 27, 2015.
- Wheelock Tim. An Introduction To Human Neuroanatomy. October 8, 2014.
- Marsh Karen R, Giffin Bruce F, Lowrie Donald J. Medical Student Retention of Embryonic Development: Impact of the Dimensions Added by Multimedia Tutorials. Anat Sci
Educ. December 2008;1(6):252-257. doi:10.1002/ase.56.
- Nicholson Daren T, Chalk Colin, Funnell W Robert J, Daniel Sam J. Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Med Educ. November 1, 2006;40(11):1081-1087. doi:10.1111/j.1365-2929.2006.02611.x.
- Petersson Helge, Sinkvist David, Wang Chunliang, Smedby Örjan. Web-based interactive 3D visualization as a tool for improved anatomy
learning. Anat Sci Educ. March 1, 2009;2(2):61-68.
- Garg A, Norman GR, Spero L, Maheshwari P. Do
Virtual Computer Models Hinder Anatomy Learning?. Acad Med J Assoc Am Med Coll. October 1999;74(10 Suppl):S87-89.
- Guillot Aymeric, Champely Stéphane, Batier Christophe, Thiriet Patrice, Collet Christian. Relationship between Spatial Abilities, Mental Rotation and Functional Anatomy Learning. Adv Health Sci Educ Theory Pract. November 2007;12(4):491-507. doi:10.1007/s10459-006-9021-7.
- Rochford K. Spatial Learning Disabilities and Underachievement among University Anatomy Students. Med Educ. January 1985;19(1):13-26.
- Estevez Maureen E, Lindgren Kristen A, Bergethon Peter R. A Novel Three-Dimensional Tool for Teaching
Human Neuroanatomy. Anat Sci Educ. December
- Kockro Ralf A, Amaxopoulou Christina, Killeen Tim, et al. Stereoscopic neuroanatomy lectures using a three-dimensional virtual reality environment. Ann Anat – Anat Anz. September 2015;201:91-98. doi:10.1016/j.aanat.2015.05.006.
- Dalgarno Barney, Lee Mark JW. What are the learning affordances of 3-D virtual environments?. Br J Educ Technol. January 1, 2010;41(1):10-32. doi:10.1111/j.1467-8535.2009.01038.x.
- Piromchai Patorn. Virtual Reality Surgical Training in Ear, Nose and Throat Surgery. Int J Clin Med. 2014;05(10):558-566. doi:10.4236/ijcm.2014.510077.
- Virtual Reality Blog. Virtual reality gear. 2009.
- Groen E, Bos J. Simulator Sickness Depends on Frequency of the Simulator Motion Mismatch: An Observation.. MIT Press J. 2008;16(6):584-593. doi:doi:10.1162/pres.17.6.584.
- Lombard M, Ditton TB, Card D, et al. Measuring Presence: A Literature-Based Approach to the Development of a Standardized Paper-and-Pencil Instrument. 9
Third Int Workshop Presence Delft Neth. 2000. .
- Lombard M, Weinstein L, Ditton TB. Measuring telepresence: The validity of the Temple Presence Inventory (TPI) in a gaming context. 2011.
- Regan C. An investigation into nausea and other side-effects of head-coupled immersive virtual reality.. Virtual Real.
- Febretti Alessandro, Nishimoto Arthur, Thigpen Terrance, et al. CAVE2:
a hybrid reality environment for immersive simulation and information analysis. In: Vol 8649. ; 2013:864903-864903-864912. doi:10.1117/12.2005484.
- Virtual Reality Applications Center. C6. The C6. 2018.
- Christie Digital Systems USA, Inc. Lights on for Mine Rescue Training. 2013.
- Tversky B. Visuospatial Reasoning. In: The Cambridge Handbook of Thinking and Reasoning. Vol Cambridge University Press; 2005:209-241.
- Hollerer T, Kuchera-Morin J, Amatriain X. The Allosphere: A Large-Scale Immersive Surround-View Instrument. August 4, 2007.
- Plunkett L. Facebook buys Oculus rift for $2 billion. 2014.
- Purchese R. Oculus answers the big Rift questions. March 9, 2014.
- Kalivarapu Vijay, MacAllister Anastacia, Hoover Melynda, et al. Game-day football visualization experience on dissimilar virtual reality
platforms. In: Vol 9392. ; 2015:939202-939202-939214.
- Engadget. Oculus VR Rift Overview. Oculus VR Rift Overv. 2014.
- Business Insider. Priceless Reactions To The Oculus Rift Virtual Reality Headset.. Youtube. 2013.
- Dargar Saurabh, Kennedy Rebecca, Lai WeiXuan, Arikatla Venkata, De Suvranu. Towards immersive virtual reality (iVR): a route to surgical expertise – Springer. May 7, 2015. doi:10.1186/s40244-015-0015-8.
- Hoffman Hunter G, Meyer Walter J, Ramirez Maribel, et al. Feasibility of Articulated Arm Mounted Oculus Rift Virtual Reality Goggles for Adjunctive Pain Control during Occupational Therapy in Pediatric Burn Patients. Cyberpsychology Behav Soc Netw. June 2014;17(6):397-401. doi:10.1089/cyber.2014.0058.
- Lin Qiufeng, Xu Zhoubing, Li Bo, et al. Immersive virtual reality for visualization of abdominal CT. In: Vol 8673. ; 2013:867317-867317-7. doi:10.1117/12.2008050.
- Hayes Dana, Turczynski Craig, Rice Jonny, Kozhevnikov Michael. Virtual-reality-based educational laboratories in fiber optic engineering. In: Vol
9289. ; 2014:928921-928921-928926. doi:10.1117/12.2070779.
- Battulga Bayanmunkh, Konishi Takeshi, Tamura Yoko, Moriguchi Hiroki. The Effectiveness of an Interactive 3-Dimensional Computer Graphics Model for Medical Education.
Interact J Med Res. 2012;1(2):e2. doi:10.2196/ijmr.2172.
- Fowler Chris. Virtual reality and learning: Where is the pedagogy?. Br J Educ Technol. March 1, 2015;46(2):412-422. doi:10.1111/bjet.12135.
- Mestre Daniel R. On the usefulness of the concept of presence in virtual reality applications. In: Vol 9392. ; 2015:93920J-93920J – 9. doi:10.1117/12.2075798.
- Tai GXL, Yuen MC. Authentic assessment strategies in problem based learning. In: Vol Singapore; 2007.
- Cook David A, Hatala Rose, Brydges Ryan, et al. Technology-Enhanced Simulation for Health Professions Education: A Systematic Review and Meta-Analysis. JAMA. September 7, 2011;306(9):978-988. doi:10.1001/jama.2011.1234.
- Chapple C. New VR treadmill the Virtualizer hits ground running on Kickstarter. July 24, 2014.
- Virtusphere. Virtusphere Product Description. 2013.
- Oblinger Diana G. The Next Generation of Educational Engagement. J Interact Media Educ. May 21, 2004;2004(1):10. doi:10.5334/2004-8-oblinger.
- Khan SA, Black JB. Surrogate embodied learning in MUVEs: Enhancing memory and motivation through embodiment. 2014.
- Freeman Scott, Eddy Sarah L, McDonough Miles, et al. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci. June 10, 2014;111(23):8410-8415. doi:10.1073/pnas.1319030111.
- Lecuyer A, Lotte F, Reilly RB. Brain-Computer Interfaces, Virtual Reality, and Videogames. 2008.
- Summers Nick. Oculus Rift Gives Medical Students a Surgeon’s Perspective. Web. August 14, 2014.
- Naaz Farah, Chariker Julia H, Pani John R. Computer-Based Learning: Graphical Integration of Whole and Sectional Neuroanatomy Improves Long-Term Retention. Cogn
Instr. 2014;32(1):44-64. doi:10.1080/07370008.2013.857672.
- Plumley Leah, Armstrong Ryan, De Ribaupierre Sandrine, Eagleson Roy. Spatial ability and training in virtual neuroanatomy. Stud Health Technol Inform. 2013;184:324-329.
- Gillner S, Mallot HA. Navigation
and acquisition of spatial knowledge in a virtual maze. J
Cogn Neurosci. July 1998;10(4):445-463.
- Richardson Anthony E, Montello Daniel R, Hegarty Mary. Spatial knowledge acquisition from maps and from navigation in real and virtual environments. Mem Cognit. July 1999;27(4):741-750. doi:10.3758/BF03211566.
- Szu-Han Wang Roger L Redondo. Wang SH, Redondo RL, Morris RG. Relevance of Synaptic Tagging and Capture to the Persistence of Long-Term Potentiation and Everyday Spatial Memory. Proc Natl Acad Sci USA 107: 19537-19542. Proc Natl Acad Sci U S A. 2010;107(45):19537-19542. doi:10.1073/pnas.1008638107.
- Skupin André. From Metaphor to Method: Cartographic Perspectives on Information Visualization. October 9, 2000.