The Science of Spatial Learning Inside Virtual Reality

Ever wondered why the global adoption of virtual reality is growing exponentially in recent years…?! If VR media was just hype for its glamour, then it might not have reached its current stage. Every other day we are seeing incredible case studies getting published which evident the benefits of virtual reality implementation in enterprise workforce training and other learning spaces. Most reports show up remarkable numbers such as up to 90% knowledge retention rate, 80% productivity increase and more.

There is a strong underlying truth which continuously fuels the growth of virtual reality technology. It is not the technical supremacy or first of its kind aspects. But the astonishing biological factors which directly influences the human brain in a much deeper way. This might look a complex subject…. but, believe me, it’s so interesting to learn about our brain using our brain…!!

Let’s get started…

Learning By Doing

“I hear and I forgot, I see and I remember, I do and I understand- Confucius, 551BC – 479BC”

 “After two weeks, the human brain remembers 10% of what it read, 20% of what it hears and 90% of what it does.” – American educator Edgar Dale – 1946

Though these famous statements are subjective and the numbers may be debatable, we can’t deny the core underlying fact that “what we learn through hands-on experience” gives utmost knowledge and wisdom. But experiential learning always comes at a cost, takes time and they also can’t be replicated or shared.

What Are Real-Experiences?

Let’s try to understand how our brain interprets real-world experiences and conceives a sense of presence. Brain understands the space and the boundary between the area around our body and what is the rest of the world. This is the space known as peri-personal space where most our interactions with the world happen; usually the space within our hands reach.

During our interaction with the physical world, various external stimuli trigger the body receptors and our sensory nerve system communicates these signals to the brain. Visual cues play a major role by providing information on light directions, depths, colors…etc.

The vestibular system in our ears helps to maintain our orientation & balance and proprioception provides a subconscious sense of where our body parts are relative to space.

The received signals are processed in our brain which has approximately 86 billion Neurons (computing nerve cell units). The processed information is then transmitted as electrochemical signals to each other via as many as 1,000 trillion synaptic connections.

Neural impulses generated in the Motor system of our brain controls the complex execution of various limbs muscle movements. The whole mental process of acquiring knowledge through our senses, body interactions, thoughts and experiences is known as cognitive learning and this the way our brain’s learning mechanism is wired by nature.

Stimulated Virtual Experiences

Let’s recall the famous dialogue from Sci-fi blockbuster “The Matrix” …. What is real? How do you define real? If you talking about what you can feel, what you can smell, what you can taste and see…then “real” is simply electrical signals interpreted by your brain.

Neuroscientists already found that virtual reality can create more believable Virtual experiences by way of artificially stimulating the receptors in our sensory systems. Designing experiences as multisensory always helps in perceiving realism in virtual reality.

Carefully crafted Virtual environments are very effective as they follow the same rules of the real world; 3D Objects in motion should obey the law of physics; Realistic shading, texturing and lighting to help us figure out volume, depth and distance of the virtual space; 360-Spatial audio, haptics and aroma also add value for realism.

Embodiment & Sense of Presence In VR Space

Embodiment in virtual reality is the key to induce the level of empathy and understanding. While designing first- and second-person VR experiences it is strongly recommended to use cognitive embodiment in a sensible way to increases the sense of presence and realism. This can be achieved by way of using virtually represented 3D avatars to inhabit ourselves (inside the virtual avatar) and virtual body parts for interactions.

This makes us feel the ownership of virtual avatars with spatial relationship with the environment.

In addition to the above physiological and psychological elements, we also have to be mindful on technical factors such as 6-DOF head & body tracking, screen resolution, antialiasing, the field of view, frame rates, latencies and navigation systems for the user to achieve a more convincing presence inside VR.

Level Of Engagement And Gamification In Virtual Reality

VR experiences should follow the rules of successful computer games where user engagement is achieved through multiple game-levels with incremental challenges. When the challenges in the learning methods are designed and incrementally adjusted stage by stage according to the Skills of the learner, the learning experience will put the brain into the Flow-State where the “Prefrontal Cortex (PFT)” partially shuts down (the main function of PFT is to control our conscious behaviors).

This puts us to the state where we loss ourselves and time and puts us within a fully pleasant learning environment a technique termed “The state of Flow”.

The famous research by Mihaly Csikszentmihalyi-Farmer Chairman, Department of Psychology, University of Chicago describes the above technique where we could learn 7 times faster than usual.

The Power Of Spatial Learning

Referring to Tom Furness one of the Godfathers of virtual reality with over 50 years of research – Tech like Virtual, Augmented and Mixed Reality leverage “cognitive embodiment” to reinforce learning. Our brain remembers places much efficiently than texts. If knowledge is provided within a 3D spatial structure, the brain memorizes and retrieves information in a very natural way where the pace of learning will be many times speedier.

University of Maryland researchers conducted one of the first in-depth analyses of the educational use of virtual reality and found that people remember information better if it is presented to them in a virtual environment.

Historical Evidence For Spatial Learning

The method of loci (loci being Latin for “places”) an imaginal technique known to the ancient Greeks and Romans is a method of memory enhancement which uses visualizations with the use of spatial memory. In this technique the subject memorizes the layout of some familiar environment such as buildings or streets which is composed of a number of discrete places.

When desiring to remember a set of items the subject ‘walks’ through these places in their imagination and commits an item to each one by forming an image between the item and any feature of that place. Retrieval of items is achieved by ‘walking’ through the places, allowing those spatial features to activate the desired items.

“Method of loci” techniques survives till today in the common English phrases “in the first place”, “in the second place”, and so forth…The efficacy of this technique has been well established and in the Brain scans of “superior memorizers”, 90% of whom use the method of loci technique, have shown that it involves activation of regions of the brain involved in spatial awareness.

Navigating In Space: The Real World

The Nobel Prize for Physiology in 2014 was awarded to Prof. John O’Keefe, May-Britt Moser and Edvard I. Moser for their interesting findings on how we remember and navigate in different places.

Our brain encodes the sense of space and navigation abilities using two types of nerve cells. When we enter into an environment for the first time, Place-cells inside the Hippocampus area of the brain are activated and a spatial map is created marking the locations.

At the same time, Grid-Cells in the “Entorhinal Cortex” part of the brain measures and records the distances between locations which provides a metric to the spatial maps in hippocampus. There are also “Head direction cells” which acts like a campus and “Border Cells” which stores the information on where the boundaries such as walls are. These complex networks within hippocampus constitute a comprehensive positioning system and act as the inner GPS for the brain.

When the spatial memory system is added inside time and few other things, the episodic memory is built which gives us the ability for remembering what we did in a particular location at a particular time in the past. Whenever we visit the same environment again, the Hippocampus enables us with easier cognitive navigation through decoding the episodic memories.

It’s also been noticed that taxi drivers have larger Posterior hippocampus as they do lot of navigations…!!

Navigating In Space: The Virtual World

The interesting finding is that the brain-cells of hippocampus fires and encodes spatial memory equally both in real and virtual environments. There are many ways virtual reality experiences can trick our brains into responding like its real life. When we move around in virtual reality, our brain creates a spatial map out of the environment so that we can navigate through it now and if we return later. Brain uses our past experiences to build a set of rules to interpret the world.

In a study published on January 2018 in the journal Nature Communications, by Center for Neuroscience, University of California: Functional-MRI was used to look for brain areas that are activated on virtual reality experience. As memories are recalled, especially in the hippocampus different regions were activated for different kinds of information.

Another interesting finding was that in this study, the hippocampus was involved in episodic memories linking both time and space of VR experiences.

The Power Of Virtual Reality

Once the brain starts to interpret the stimulated signals as real then these Virtual experiences will become the ultimate weapon to break all those limitations of the Physical-Reality. Through VR, we can virtually bring the physical places inside our brain and tap the highest possible learning potential which was not possible before.

In the neuro-medical treatment known as neuroplasticity, recently VR is being used extensively to rewire the neural network connections of the brain. This technique establishes new connections and functionalities in our brain which might otherwise be lost by the natural metabolic process known as “Synaptic pruning “and untapped throughout our life.

VR devices can also quickly learn about how the users respond to different stimuli by tracking their eye movements and head positions and even monitor their emotional states through EEG. This whole new operating system for our brain will let us learn, feel, remember and process new ideas in a more experiential way. Decision making will become through empathy and data and not just by intuition.

Postface

Developers have to be mindful about the fact that most nauseated users after a bad VR experience will immediately conclude the VR-media is always bad rather than complaining about the content. Over the period, this will poison and ruin the future of this wonderful media.

Creating a completely compelling VR may be practically challenging and in most cases, the effort spent on attempting towards highest perfection may not really worth it. Through experience, the better-informed VR creators can soon figure out the balancing recipe while maintaining the fidelity of the virtual presence.

VR-developers should also realize their biggest social responsibility while conceiving concepts for VR-games since such prolonged experiences will potentially transform the users to behave violently even in the real world. It’s our responsibility to understand and accept the technological revolution to leverage its unique potentials in the right way.

Note: We at Inc42 take our ethics very seriously. More information about it can be found here.