Reality Check

Virtual Reality, Real-World Results

Remember Pokémon GO? Released in 2016, the interactive video game made cartoon characters appear (or appear to appear) in real-world locations via a smartphone app.

“That was a huge cultural phenomenon,” says Jack Wolfe, academic media technologist at Bentley and an expert in extended reality (XR) technology.

But the Pokémon GO craze was also short-lived. Which might lead one to believe XR was a fleeting novelty with few applications outside of gaming. And nothing could be further from reality.

Extended reality is an umbrella term that includes both virtual reality (VR) and augmented reality (AR). “Augmented reality — sometimes called mixed reality — is when you can see the world around you, but artificial graphics are overlaid on your vision,” Wolfe says. Pokémon GO, for example. “Virtual reality is when you’re fully immersed in an environment such as a game or an app, usually through some kind of head-mounted display.”

And the technology offers tantalizing possibilities for real-world use cases in many fields. The most obvious is its potential to solve the “How do you practice that?” problem. Certain occupations, such as piloting airplanes and performing surgery, require high-risk procedures with no margin for error. That makes attempting them for the first time a harrowing prospect.

“One of the most compelling developments in VR is its ability to train people such as first responders for stressful situations that are hard to replicate,” says Stephan Demers ’09, manager of network design and logistics compliance at Meta Reality Labs, who double majored in Computer Information Systems and Global Perspective on Commerce and Culture. “Now we can do it for lots of different jobs without having to actually recreate the physical environment.”

Realizing Remote Possibilities

Extended reality technology can also be used in industrial environments.

Tejas Shroff ’16 worked with XPO Logistics, a freight-shipping specialist, on a VR training application that enhanced employees’ awareness of potential hazards. “It helps people identify safety concerns and develop proper protocols on the warehouse floor before they actually step onto the warehouse floor,” he says. So when new hires actually have to “load trucks or shift heavy pallets around in an environment where there’s a lot of heavy machinery,” he explains, they will already be oriented and know what to watch for.

Another benefit of XR technology is that it enables remote troubleshooting. For example, technicians at high-end European car manufacturers can now provide support to their counterparts in the U.S. who are wearing camera-equipped AR glasses. “The expert in Germany can see what the tech is seeing and instruct them in real time on how to problem-solve or do an install,” Demers says.

Advances in lightweight, wearable AR technology — including smart glasses that offer many of the same features as smartphones but with hands-free use — are rapidly expanding the possibilities under development. “The most exciting one for me is real-time translation,” Demers says. “As someone who often spends six to 10 weeks a year out of the country, it’s like a wall coming down in terms of being able to communicate.” It’s kind of like science fiction coming to life, he says: “Real-time translation has been in the air for 60 years, going back to Star Trek, and now you can actually buy it for $300.”

Staying Ahead of the XR Curve

Another form of XR technology is “cinematic reality,” or CVR, which is VR created with cameras rather than computer-generated images. “With a CVR experience, you don’t need to build the environment,” says Steve Salina ’91, P ’19 and ’21, Bentley’s principal instructional and research media production and infrastructure engineer. “You just need to put a camera in a specific environment and capture it.”

Using 360-degree underwater cameras, Salina collaborated with Betsy Stoner, assistant professor of Natural and Applied Sciences, to develop a “virtual field trip” for Stoner’s class on Ecosystem Services in the Caribbean. “We ended up getting a bunch of things that we didn’t expect to get, like sea turtles,” he says. “They swam right up to the camera.”

The result is an experience that’s immersive in more ways than one. Students feel like they’re underwater. (Salina notes that one student actually ducked when the sea turtles swam toward the camera.) That vividness improves their ability to retain information. “That’s why it’s so valuable,” he says. “It’s not just the wow factor — they’re actually learning something.”

Still, XR’s wow factor is pretty awesome. Demers felt it when he first visited Meta’s Reality Labs and was treated to a demo of their advanced VR technology. “It was a walk-through of the International Space Station,” he recalls. “That was the first time I was in an immersive virtual world. I remember walking up to the window of the space station and looking down at Earth — and feeling goose bumps on my arms. That’s when I realized this was a chance to be a part of something that was going to be truly transformational.”

The Soft Sell

XR also offers the potential to improve development of so-called “soft skills.” Silke Plesch, a senior lecturer in Natural and Applied Sciences who teaches cyber psychology courses, says this could fill a growing void. “Soft skills,” she says, “are actually really hard.”

The pandemic and its resulting isolation made many students even more dependent on technology to communicate. With that in mind, Plesch and her Bentley colleagues have developed XR training that uses AI-powered avatars equipped with ChatGPT. Students can then practice communication skills such as debate, negotiation and conflict resolution “without the pressure of performing in front of 28 other students in a classroom,” she says.

“Gen Z are all growing up with technology at their fingertips 24/7, so it’s important that we have the latest technology available in the classroom so we can meet them where they are. A traditional classroom environment with a professor just standing in front of them and lecturing away isn’t going to meet the students’ learning needs through engaging with technology.”

And, adds Gaurav Shah, MBA ’01, MSHFID ’12, Bentley’s senior director of academic technologies and user experience design, it’s something that employers need to take into consideration as well. “Many companies use VR headsets to train managers in having difficult conversations in which they have to provide negative feedback or for HR applications involving sensitive topics such as sexual harassment,” he says. “AR/VR enables supervisors to practice so they can get comfortable before having those conversations in the workplace.”

With the global augmented reality market size expected to increase from roughly $140 billion today to $1.7 trillion by 2032, XR education and training are growing in importance — for students, current employees and employers. At Bentley, these new technologies are worked into learning and development in a range of applications.

Take, for example, virtual “international cafes,” which enable students to familiarize themselves with different languages and cultures. “That’s great for MBA students who will be interacting with people on a global level,” Salina says. Or being able to pitch an entrepreneurial idea to potential “investor” avatars. In a university setting, perhaps the most valuable aspect of XR is its ability to simulate actual reality.

“That’s part of what Bentley prides itself on,” he continues. “When students come out of here, they’re prepared when they encounter various situations in the real world because they’ve already experienced them in school.”

A look at just some of Bentley’s classes using AR and VR.