Augmented reality (AR) is a relatively new technology that’s gained increasing popularity in recent years. It enables users to interact with digital content in a physical environment in real-time, offering an enhanced experience. This technology has the potential to revolutionize many industries, from entertainment to education, by providing a more realistic way to interact with digital content.
The underlying technology for AR is complex and combines custom-built hardware with advanced machine learning techniques. As the software and hardware improve, we can expect augmented reality to grow in popularity until it becomes part of everyday life.
The technology behind augmented reality is a culmination of many years’ research and development. Most of the technologies take root in computer vision where they’ve been used to solve individual problems. Augmented reality combines many of these technologies in a complex way, and must do so in real time.
How Does Augmented Reality Work?
- Sensing: In order to combine the physical world with digital objects, the device must have a model of the physical world. This is achieved through a wide array of sensors. Typical sensors include standard cameras (light sensors), depth sensors, gyroscopes and GPS.
- Modeling: With the sensory information as input, the device must then process this information to obtain a model of the world. This will include a 3D positional model but will also contain many more characteristics. Typical machine learning models applied here are instance segmentation and object detection.
- Enhancing: AR’s power is in the way it enhances our environment, which is what happens in this step. The device enhances the model obtained in the previous step to create a combination of physical and digital. The technology achieves this through the addition of software assets, which can range from simple counters to interactive dinosaurs.
- Display: Finally, the device must display the model it has created to the end user. This is commonly through a display on a smartphone or dedicated headset.
Types of Augmented Reality
Within this broad framework of augmented reality, there are distinctions between various types of augmented reality. These include marker vs. marker-less technologies and digital vs. passthrough technologies.
Marker vs. Marker-Less AR Technology
In marker-based AR, we use a specific marker as the basis for the modeling. For example, we can use a QR code on a wall to project an artwork. The device knows the expected size and shape of the QR code, and knows it is a flat position on the wall, which makes it significantly easier to project onto. Marker-based technology is also more limited due to the constraints of the markers. On the other hand, marker-less technology must have a wider set of sensors and a more complex understanding of the environment. This gives it the flexibility to display many different scenarios.
Digital vs. Passthrough AR Technology
In digital AR technologies, the world is measured through cameras and then projected back to the user on a screen. This digital reproduction comes with a host of issues, including fidelity, latency and screen fatigue. In passthrough technologies, the user is able to see the physical world directly, with digital overlays on top. For example, imagine a mirror-based projection where the user can see through glass and a well-positioned projector bounces light off the glass to overlay digital information.
Examples of Augmented Reality
While augmented reality is still very much at the beginning of its growth, there are many examples of AR in the real world from entertainment to e-commerce to everyday work. This technology makes possible exciting and practical ways to interact with our environment.
Photo and Video Viewing
Viewing photos, videos or live streams becomes a more immersive experience as users can place, fix and adjust these visuals within a physical environment. The Apple Vision Pro headset by Apple helps to do just this. While wearing the headset, users can put up 2D images and videos in a 3D space, and see a photo album or a FaceTime call as if it’s displayed on the wall or in mid-air in their home.
On-The-Job Training
The first big public display of augmented reality technology was Google Glass, which made headlines after employees were spotted testing it out in public. It’s an example of passthrough AR technology, which projects digital overlays to the user through a mirrored display. While it never made it to the consumer market, Google Glass still has healthy usage in commercial applications including inventory management and training.
E-Commerce
Closer to consumers, there are numerous examples of virtual “try-on” solutions offered by fashion retailers, such as Farfetch and Prada. Users can pose in front of their smartphone and have new styles projected right onto their body. This gives buyers a much better sense of how clothing items will appear if they were to purchase and helps companies to increase product discovery.
Gaming
The most widely played AR game is Pokémon Go with over 200 million players. In this game, players can use their smartphone to see their surrounding map covered in wild Pokemon. They’re able to take it further and project the Pokemon onto their surroundings using the camera function. This is a great example of how AR technology can be used to create a unique and engaging game.
Meta Quest vs. Microsoft HoloLens
We can find more advanced applications of augmented reality in Meta’s Quest and Microsoft’s HoloLens. Both of these companies produce high-end headsets with augmented reality technologies. Similar to Google Glass, these devices have business applications which make them enticing to larger corporations. Meta’s Quest headset supports many standard business functions such as meeting collaboration and multi-monitor displays. The HoloLens has been tested for medical training as well as military applications (though with delayed results for the latter).
Augmented Reality vs. Virtual Reality
While augmented reality and virtuality reality (VR) sound related, they are very different technologies. In VR technology, the user becomes fully immersed in a virtual world and experiences feeling detached from the physical world around them.
VR technology is easier to develop because it doesn’t require reproduction of the user’s environment. This restricts the potential applications but the relative simplicity of the technology means there’s a much deeper set of content currently available commercially, mostly in entertainment. That said, while you can recreate a flight cockpit in VR and have a pilot train in the virtual environment, it will always be an inferior experience compared to the pilot sitting in a real cockpit with physical switches, and even having guided experience during a flight.
The Future of Augmented Reality Technology
The future of AR is incredibly exciting and holds tremendous potential for how we interact with the world around us. With the rapid advances in computing power and the proliferation of affordable hardware, the possibilities are seemingly endless. Picture your home overlaid with real-time data and information, digitally changeable artwork and wallpapers, and technology that helps you decide where that new couch can go. Additionally, with the hype building around the metaverse, augmented reality technology is set up to become the tool we use to interact with it in many ways.
AR technology will become a ubiquitous part of our lives, providing us with the tools to create, explore and discover ideas in ways we never thought possible... or imagined only as sci-fi fantasy.
