Are virtual reality and augmented reality the future of education?

Welcome to the memory palace; and how to safely walk around (or maybe run) in VR
June 19, 2018

People learn better through virtual, immersive environments instead of more traditional platforms like a two-dimensional desktop computer: That sounds intuitively right, but researchers at the College of Computer, Mathematical, and Natural Sciences at the University of Maryland (UMD) have now supported that idea with evidence.

The researchers conducted an experiment using the “memory palace” method, where people recall an object or item located in an imaginary physical location, like a building or town. This “spatial mnemonic encoding” method has been used since classical times, taking advantage of the human brain’s ability to spatially organize thoughts and memories.*

Giulio Camillo’s Theater of Memory (1511 AD)

The researchers compared subjects’ recall accuracy when using with a VR head-mounted display vs. using a desktop computer. The results showed an 8.8 percent improvement overall in recall accuracy using the VR headsets.**

The two virtual-memory-palace scenes used in the experiment (credit: Eric Krokos et al./Virtual Reality)

Replacing rote memorization

“This data is exciting in that it suggests that immersive environments could offer new pathways for improved outcomes in education and high-proficiency training,” says Amitabh Varshney, PhD., UMD professor of computer science. Varshney leads several major research efforts on the UMD campus involving virtual and augmented reality (AR), including close collaboration with health care professionals interested in developing AR-based diagnostic tools for emergency medicine and in VR training for surgical residents.

“As Marshall McLuhan pointed out years ago, modern 3-D gaming and particularly virtual-reality explorations, like movies, are ‘hotter’ than TV, while still requiring participation,” said Western Michigan University educational psychologist Warren E. Lacefield, Ph.D. “That greatly enhances the ‘message,’ as this formal memory study illustrates with statistical significance.”

Inside the Augmentarium***, a revolutionary virtual and augmented reality facility at UMD (credit: UMD)

Learning by moving around in VR

Other research suggests the possibility that “a spatial virtual memory palace — experienced in an immersive virtual environment — could enhance learning and recall by leveraging a person’s overall sense of body position, movement and acceleration,” says UMD researcher and co-author Catherine Plaisant. A good example is the obsessive AR game Pokémon GO, which combines the fun and hyperfast learning one doesn’t experience in conventional education.

While walking (or running around) in VR sounds like fun, it’s currently a challenge (ouch!). But computer scientists from Stony Brook University, NVIDIA and Adobe have developed a clever way to allow VR users to walk around freely in a virtual world while avoiding obstacles. The trick: the system makes imperceptible small adjustments in your VR-based visual reality during your blinks and saccades (quick eye movements) — while you’re (unknowingly) totally blind.

Eye fixations (in green) and saccades (in red). A blink fully suppresses visual perception. (credit: Eike Langbehn)

In other words, it messes with your mind. For example, a short step in your living room could make you believe you’re taking a large stride in a giant desert or planet — without knocking over the TV.

That system will be presented at SIGGRAPH August 12–16 in Vancouver. Watch out for the VR walkers.

Ref. Virtual Reality (open access), ACM Transactions on Graphics (forthcoming). Source: University of Maryland, SIGGRAPH, Stony Brook University, NVIDIA, Adobe.

* A remarkable example, described here, is a Russian who was thought to have a literally limitless memory.

** For the study, the UMD researchers recruited 40 volunteers—mostly UMD students unfamiliar with virtual reality. The researchers split the participants into two groups: [to eliminate one viewed information first via a VR head-mounted display and then on a desktop; the other did the opposite. Both groups received printouts of well-known faces—including Abraham Lincoln, the Dalai Lama, Arnold Schwarzenegger and Marilyn Monroe—and familiarized themselves with the images. Next, the researchers showed the participants the faces using the memory palace format with two imaginary locations: an interior room of an ornate palace and an external view of a medieval town. Both of the study groups navigated each memory palace for five minutes. Desktop participants used a mouse to change their viewpoint, while VR users turned their heads from side to side and looked up and down.

Next, Krokos asked the users to memorize the location of each of the faces shown. Half the faces were positioned in different locations within the interior setting—Oprah Winfrey appeared at the top of a grand staircase; Stephen Hawking was a few steps down, followed by Shrek. On the ground floor, Napoleon Bonaparte’s face sat above majestic wooden table, while The Rev. Martin Luther King Jr. was positioned in the center of the room.

Similarly, for the medieval town setting, users viewed images that included Hillary Clinton’s face on the left side of a building, with Mickey Mouse and Batman placed at varying heights on nearby structures.

Then, the scene went blank, and after a two-minute break, each memory palace reappeared with numbered boxes where the faces had been. The research participants were then asked to recall which face had been in each location where a number was now displayed. The key, say the researchers, was for participants to identify each face by its physical location and its relation to surrounding structures and faces—and also the location of the image relative to the user’s own body.

The results showed an 8.8 percent improvement overall in recall accuracy using the VR headsets, a statistically significant number according to the research team. Many of the participants said the immersive “presence” while using VR allowed them to focus better. This was reflected in the research results: 40 percent of the participants scored at least 10 percent higher in recall ability using VR over the desktop display.

*** The  Augmentarium is a visualization testbed lab on the UMD campus that launched in 2015 with funding from the National Science Foundation. The researchers describe it as “a revolutionary virtual and augmented reality (VR and AR) facility that brings together a unique assembly of projection displays, augmented reality visors, GPU clusters, human vision and human-computer interaction technologies, to study and facilitate visual augmentation of human intelligence and amplify situational awareness.”