How VR and AR Are Used in Surgery: 12 Examples

Virtual and augmented reality aren’t just for video games. Check how these VR and AR companies are driving surgical innovation.

Written by Sam Daley
How VR and AR Are Used in Surgery: 12 Examples
Image: Shutterstock
UPDATED BY
Brennan Whitfield | Nov 17, 2022

Virtual reality (VR) and augmented reality (AR), first popularized by the gaming industry, have made their way into healthcare. Now, VR and AR tools can display interactive 2D and 3D images of physical procedures or anatomy, helping surgeons to maximize efficiency and minimize risk in the operating room.

What Is VR and AR Surgery?

Virtual reality (VR) and augmented reality (AR) are simulated imaging tools used in surgery. Using 2D and 3D image rendering from VR/AR tools, surgeons can mimic real surgical procedures and anatomy, boosting preparedness and efficiency in the operating room.

 

Benefits of VR and AR in Surgery

VR and AR Surgery Benefits

  • Access to 3D anatomical images and models.
  • Access to real-time medical references and information.
  • Accelerates surgical operation prep times.
  • Provides low-risk surgical testing environments.
  • Reduces surgical equipment and staff costs.

With VR and AR in surgery, surgeons have pre-operation access to 3D renderings of hearts, eyes, knee joints and lots else. In some cases, surgeons even use AR-enabled haptic feedback gloves to mimic the buzzing of saws and drills. All of these resources help to prep for operations, study realistic models of patients and access real-time anatomical references and information.

Besides being a boon to surgical preparation, these extended reality (XR) tools also help hospitals save time and money. The ability to study 3D scans before an operation helps surgeons thoroughly prep for each case and implement proactive, time-saving procedures. Another plus: storing images and patient data on one AR/VR platform reduces the need for expensive screens and unnecessary staff.

 

12 Companies Using VR in Surgery

1. Vicarious Surgical

Location: Waltham, Massachusetts 

Vicarious Surgical’s robotic system, controlled by a VR headset, is designed to allow freedom of movement and precision in minimally invasive surgeries. The surgical robot works by mimicking upper-body movements and using a 360-stereoscopic imaging hardware piece for its “head.” This piece connects to a VR device and is fully responsive to user movements, providing an inside view of a surgical site as if the surgeon was physically there.

 

2. Augmedics

Location: Arlington Heights, Illinois   

With an AR surgical guidance system, Augmedics grants surgeons “X-ray vision” capabilities during spinal procedures. The xvision Spine System uses an AR headset integrated with a patient’s intraoperative data, which can display 3D skeletal models that follow the surgeon’s view. By using this technology, Augmedics aims to eliminate surgeon distractions and reduce traditional X-ray radiation for the patient. 2,000 commercial cases using the xvision system have been completed as of 2022.

 

3. EchoPixel

Location: Los Altos Hills, California

The EchoPixel interactive virtual reality surgery platform helps doctors to identify anatomy of interest and allows them to use the platform’s depth information to enhance detailed medical images in real-time. The EchoPixel True3D platform uses a wide variety of image datasets to produce anatomically correct, patient-specific 3D images. The company made international news for its 3D images of colons that help surgeons plan operations and produce real-time analytics.

 

4. FundamentalVR

Location: Fully Remote 

FundamentalVR delivers haptic simulators that let surgeons practice and sharpen their skills. Using realistic vibration patterns, the haptic technology gives doctors the feeling of holding actual tools. Combined with VR, the FundamentalVR ecosystem provides doctors with a lifelike platform to study surgeries and practice them. FundamentalVR has worked with well-known leaders across the health and tech industries, including Amazon, Oculus, Bayer and University College London Hospital.

 

5. ImmersiveTouch

Location: Chicago, Illinois

ImmersiveTouch creates virtual reality solutions for surgical planning, training and education. The company’s ImmersiveView Surgical Plan platform generates 3D replicas from patient scans, allowing surgeons to study and collaborate with their team on surgical tactics. Using the Oculus Rift headset, surgeons are able to use a number of tools, like the cut, draw and measure tools, that emulate a real procedure. Johns Hopkins, University of Chicago and University of Texas hospitals have all used the ImmersiveTouch platform to assist their surgeons in studying and planning surgeries.

 

6. Medical Augmented Intelligence

Location: Fairfax, Virginia

Medical Augmented Intelligence creates AR and VR educational tools for doctors and medical students. The company’s Body Map helps students and professionals observe an intricately detailed, anatomically correct model of a human body so they can learn about different systems and practice their surgical skills. Doctors at the Chang Gung Proton Therapy Center have used Augmented Intelligence’s AR tools to map a patient’s optimal lying position to administer proton therapy. Employing AI and algorithms, the center can treat a patient’s cancer more quickly and efficiently.

Find out who's hiring.
See jobs at top tech companies & startups
View All Jobs

 

7 . Osso VR

Location: Boston, Massachusetts

Osso VR is a virtual surgery simulation platform that offers realistic hand-based interactions. The company’s VR focuses on practicing with the virtual tools that surgeons use for orthopedic and spinal surgeries. Osso VR has also developed an interactive platform where multiple surgeons can communicate with one another and go through training lessons or practice procedures simultaneously.

 

8. Proprio

Location: Seattle, Washington

Proprio combines machine learning and AR to create ultra-precise 3D medical images. The imaging tools help surgeons see through obstructions and collaborate with colleagues on surgery plans. The built-in AI platform also analyzes the 3D rendering and stores and shares surgical data in real-time. Proprio runs pilot programs in neuro and orthopedic surgeries in hospitals like Seattle Children’s and University of Washington Medicine.

 

9. Proximie

Location: Boston, Massachusetts

Proximie’s suite of AR tools help doctors find ailments in patients and describe surgeries using 3D models. They’re able to scan a patient’s body, point out health problems (like tumors or broken bones) and use the images to show patients a step-by-step process of their potential surgery.

 

10. SentiAR

Location: Saint Louis, Missouri

SentiAR is a holographic augmented reality platform for intraprocedural clinical use. While they’re performing a procedure, surgeons can view a 3D image of specific anatomy (it floats above the patient on the operating table) in real-time. The visualization is hands-free and gives surgeons in-depth insights about a patient’s health throughout the surgery. The SentiAR platform has been successful in treating and analyzing cardiac arrhythmias in a lab environment.

 

11. Surgical Theater

Location: Los Angeles, California

Surgical Theater created a VR-based Surgical Rehearsal Platform (SRP) for neurosurgical pre-op planning and rehearsal. The company’s platform scans 2D brain images and on-site VR specialists create individualized 3D models. The brain models are then used to help guide a patient through the surgery details and as a planning tool for the surgical team. NYU, the Mayo Clinic and UCLA all have used Surgical Theater’s VR brain modeling to plan surgeries and thoroughly explain the procedures to patients.

 

12. zSpace

Location: San Jose, California  

ZSpace creates hardware and software solutions to support the use of VR and AR for various industries. Its health science applications let users explore 3D human biology models, surgical and equipment safety procedures as well as over 4,600 anatomical structures in a VR environment. Zspace also offers professional development and online learning services for using the technology in the classroom or workplace.

Hiring Now
InStride
Edtech • Enterprise Web • Social Impact • Software
SHARE