Mixed reality blends elements of augmented reality (AR) or virtual reality (VR) with a user’s physical surroundings to create an immersive experience rooted in both the physical and virtual world.
What Is Mixed Reality?
Mixed reality combines augmented reality or virtual reality elements with the physical world to create an immersive experience.
While virtual reality immerses users in a simulated 3D environment, and augmented reality layers elements of the virtual world upon real-world surroundings, mixed reality combines the two to create an experience where users can interact with both the virtual and physical worlds more seamlessly.
The same headsets we use now to access the virtual world are, and will likely continue to be, used to access mixed reality experiences. Though there will be some tweaks — like enhanced video passthrough capabilities, which essentially allows video to be layered over the physical world in front of you — much of mixed reality will be defined by applications that can effectively blend physical surroundings with virtual settings created by the likes of Apple, Meta, Microsoft and others.
VR vs. AR vs. Mixed Reality
Though working off the same idea of interacting with simulated elements in a physical space, VR, AR and mixed reality all involve different environments and mechanics for their technologies. Here’s a breakdown of each and how they compare.
Virtual Reality (VR)
Virtual reality describes a visual, immersive, often 3D experience that provides users the feeling of being placed in a digital world.
VR requires users to wear a VR headset, a goggle-like headpiece which goes over the eyes and displays a screen of the experience directly in front of users. These headsets apply near-eye display and pose tracking, which allow users to control in-VR actions and elements with their own real-life movements. If a user moves their head to the left in the real world, their view in the VR world will also move left.
VR hardware can also employ handheld controllers, which can let users pick up items or use tools in the VR environment based on their hand movements.
Augmented Reality (AR)
Augmented reality describes the application of digital elements — like images, animations and sounds — on top of a real-world environmental view.
Unlike VR, AR doesn’t require any wearable hardware to achieve its effect. AR is often experienced through a smartphone and its camera technology, as computer-generated visuals are superimposed onto a physical world view captured by the camera. From here, users can move and tap their screens to interact with these environments through a 2D view.
An example of AR can be seen in the mobile video game Pokémon Go. As players travel around in real life, Pokémon creatures will appear based on different areas, and can be seen in digital form and “caught” through a player’s smartphone.
Mixed Reality (MR)
In mixed reality, VR and AR aspects are combined. This means that objects and actions from the real world can affect simulated objects in an MR environment.
Mixed reality requires the use of a headset like for VR, though often these headsets are also equipped with a camera so it can scan a user’s surroundings. The result is being able to see the physical world through your VR headset, but with superimposed visuals and effects.
For example, while standing at a physical table with an MR headset on, a virtual block could emerge on your screen and appear to be sitting on the table as well. With your real-life hand, you could then appear to push the block, hold it or knock it off the table completely through your headset view.
Mixed Reality Use Cases
Today, mixed reality is being implemented in a wide range of settings. Mixed reality applications are being used to teach workers how to assemble doors for airplanes, to let medical students practice in an operating room setting and to turn living rooms into escape rooms.
GAMING
VR technology has already made waves in gaming, paving the way for mixed reality gaming to emerge. With mixed reality, players can fight enemies, traverse obstacle courses or plant a virtual garden — all taking place inside their living room.
Some of the biggest names in tech, such as Meta and Microsoft, are already making an impact in the MR gaming industry with products like the Meta Quest series and Microsoft HoloLens series.
Healthcare
Mixed reality in healthcare could be used to train students or for immediate resource access in time-sensitive scenarios like surgeries.
In a learning environment, students could use mixed reality to follow visual demonstrations of care steps and apply them in real-time on a training model. In a surgical environment, surgeons could utilize MR to scroll through patient documents or even view inner anatomy without having to physically interact with the patient.
Retail
In retail, mixed reality is creating a more engaging experience for shoppers.
VNTANA, a company that creates 3D and augmented reality infrastructure platforms, uses algorithms to optimize a brand’s 3D designs of merchandise. This could apply to running shoes or handbags, so they can be scaled and used anywhere from Facebook to mobile websites to other virtual platforms like digital showrooms and the metaverse, Ashley Crowder, CEO and co-founder, told Built In.
On the consumer side, 3D modeling is becoming increasingly popular, giving shoppers a clearer understanding of products while shopping online, Crowder said. Three-dimensional content, she added, has shown to increase conversion rate and average cart size while reducing returns and unlocking a new revenue stream.
Sports
According to a report from technology intelligence company ABI Research, 3D content is playing a crucial role in the leadup to the metaverse and the “merging of the real and virtual worlds.”
Today, these increasingly immersive experiences are also being tested in organized sports by companies like Arcadia.tv, a mixed reality competitive sport platform that allows users to compete across both the virtual and physical worlds.
“It’s not only a video game world — it’s not limited by reality,” Chris Olimpo, Arcadia’s CEO and co-founder, told Built In.
“It’s not only a video game world — it’s not limited by reality.”
These two worlds blend together, thanks to the large roller hockey rink Arcadia is using to stage obstacle course-like competitions for players donning virtual reality headsets. It’s a large space, one where participants can run without fear of running into walls, while breaking a sweat, having fun and experiencing a technology that offers a true mix of both virtual and physical worlds.
Workplaces
Andrew Hawken, CEO and co-founder at Mesmerise, a virtual and augmented reality platform that’s used for meetings, conferences, events and training, said his company is helping companies migrate to the metaverse. These enterprises are making this leap with virtual reality in hopes of engaging their workforce — many of whom have never been to the office — whether through onboarding, training or other company events.
According to Hawken, training or hybrid work rooted in VR is helping workers experience what it’s like to have their work community reunited. And he believes the next generation of headsets will enable mixed reality even further in office spaces. “I think it’s going to be a big part of everyone’s working lives,” Hawken said. “I think there will be scenarios where these headsets just become the norm alongside your phone or laptop.”
“I think there will be scenarios where these headsets just become the norm alongside your phone or laptop.”
“I think there will be scenarios where these headsets just become the norm alongside your phone or laptop.”
Apple’s mixed reality headset, Apple Vision Pro, is slated for release in early 2024. Meta released its Meta Quest Pro headset in October 2022, and is releasing Meta Quest 3, its next mixed reality headset in the Quest series, in Fall 2023.
As more mixed reality tools, like applications that allow workers to collaborate with one another in the metaverse, and headsets become available, this blend of the digital and physical may become more commonplace in offices and other work settings.
What Role Will Mixed Reality Play in Our Lives?
While virtual, augmented and mixed reality will be more widely adopted and integrated into everyday lives — for entertainment, work and socialization — it’s still unclear just how big a role they’ll play.
When it comes to the metaverse, mixed reality applications and hardware are poised to have an impact. Winston Robson, CEO and Founder of WeMeta, an analytics tool for managing digital land in the metaverse, sees mixed reality offering a range of experiences in the metaverse today, from work to one-off events, though there’s a lingering problem around the creation of engaging, compelling content.
And hardware remains an issue.
“Use cases can sometimes be a challenge because you’re going to be wearing this thing on your face all the time,” Olimpo said. “And that’s a hard thing to fathom.”
This is one of the big reasons why Olimpo believes sports is the best outlet for mixed reality applications. Many athletes have something strapped to their head during the course of a game and for a relatively short period of time. But not only that, it’s also the physical activity inherent in Arcadia that makes these mixed reality competitions so valuable.
“My big dream here is that one day parents are going to tell their kids, ‘Why don’t you go outside and play video games already?’”
“I do think the main benefit on our side is getting society out of their chairs, getting people moving, getting people wanting to go to a shared space and build a true community,” Olimpo said.
He envisions a day when technology once again brings people together, much like movie theaters did before most people had their own version of a movie theater in their living room.
But what would it take to get there?
Low-cost and accessible mixed reality headsets and applications could be the answer. As these technologies become more ubiquitous, Olimpo hopes a day comes where kids are able to run down to the neighborhood park with headsets in hand and start playing with their friends in a mix of both the physical and virtual worlds around them.
“My big dream here is that one day parents are going to tell their kids, ‘Why don’t you go outside and play video games already?’”
