YouтАЩve heard of virtual reality, but how about augmented reality? The name might sound unfamiliar, but chances are you may have used it and not realized it. You may have even played a game thatтАЩs built on it. A popular example is Pok├йmon Go, a smartphone app that lets you go around your neighborhood or community тАЬcatchingтАЭ wild Pok├йmon that look on the screen as though theyтАЩve popped up right in front of you.
Virtual reality and augmented reality are closely related. In both, you can take part in a computerized virtual world that appears in front of you, whether itтАЩs on your phone screen or a headset.
But thereтАЩs one really important difference. Virtual reality is immersive. That means it allows you to dive in to a fully make-believe world, with sight and sound. While you can engage with it and it might even look realistic or lifelike, virtual reality exists entirely apart from anything real.
Continued
What makes augmented reality special and more useful in medicine is that it merges virtual images with the real world, including real objects and real people. That means your surgeon, doctor, or nurse could see things in front of them using augmented reality that they otherwise couldnтАЩt — like the veins running through your arm, a broken bone, brain tumor, X-ray, or health records — without taking their attention away from you to look at some other screen.
тАЬ[In augmented reality], thereтАЩs something floating in from of you in the room that youтАЩre in,тАЭ says Greg Dorsainville, manager of immersive computing at NYU Langone Health. тАЬYouтАЩre in your world, and data is overlaid on top of items in the world.тАЭ
Augmented Reality in the Clinic
As the technology improves, augmented reality has lots of potential to impact medicine. Augmented reality may find its way into the primary care clinic, operating room, emergency room, and dental office. Doctors could use it, for example, to plan plastic surgeries and other complex operations. They also could use it to help guide them during surgeries of various kinds.
Continued
Augmented reality involves the use of software technology to enhance the real-world environment, says Thomas Hopkins, MD, chief medical officer for a company called AccuVein. In health care, thereтАЩs a lot of opportunity to improve the technique to educate people, make procedures better, and to take care of people.
AccuVein is one example. The handheld device uses a laser-based technology to тАЬseeтАЭ through your skin and into your veins. ItтАЩs intended to make it easier for doctors, nurses, or others to find a vein to draw blood or place an IV. Hopkins, whoтАЩs also an assistant professor of anesthesiology at Duke University Health System, says thereтАЩs already some data showing that the augmented reality device makes things easier for both doctors and the people theyтАЩre treating. Health providers are more likely to get it right the first time, lowering the risks of extra needle pokes while saving time.
Augmented Reality in the Operating Room
Augmented reality is just starting to find its way into higher-stakes medical care. In June 2020, neurosurgeons at Johns Hopkins University in Baltimore announced the first augmented reality surgery. A doctor used it to help place six screws during a spinal surgery to help with serious back pain. Soon after, they used it to remove a spinal tumor from someone else.
Continued
The augmented reality technology included a headset with a display the doctors could see through to the person. It allowed them to project images from X-rays or CT scans, for example, onto the body, to see both at the same time. As long as those images are lined up just right, itтАЩs as if surgeons have X-ray vision.
Timothy Witham, MD, the Johns Hopkins neurosurgeon who led the surgeries, says the technology acts something like GPS, guiding the way in the operating room. The alternative is for doctors to do their best to place screws тАЬfreehand.”
тАЬThere are pros and cons to all techniques,тАЭ he says. тАЬThe main pro [of doing it freehand] is itтАЩs fastest. But it requires the best knowledge of anatomy and experience doing it almost blindly based on anatomic landmarks.тАЭ
Continued
ItтАЩs possible to use an X-ray, but that means radiation exposure for you and your surgeons. Witham says heтАЩs now using augmented reality tools routinely instead. HeтАЩs found it allows for 98% accuracy, which is at least as good as any other method.
Continued
тАЬI feel more comfortable when I use it,тАЭ he says. The anxiety level is down a notch in terms of worrying about where to put the tools. ThereтАЩs added confidence to doing it, and thatтАЩs helpful. Ultimately, that benefits patients.
One reason for its early use in spinal surgery is that the spineтАЩs rigid structure makes it easy to link computerized images correctly with the body. Witham says it would be harder to use for operations on other areas like the belly or chest, where movement makes it much trickier to keep the virtual and the real worlds aligned.
Augmented Reality in Medical Education
While many applications in medicine arenтАЩt ready yet, augmented reality is already in regular use for doctors and nurses in training.
тАЬIf itтАЩs not tracking correctly [in the operating room], you would have errors,тАЭ Dorsainville says. тАЬIn teaching and learning, you donтАЩt need that level of precision.тАЭ
At NYU Langone, students and faculty use augmented reality tools to access learning materials and learn human anatomy. For example, they can spin a detailed 3D model of the heart around on a computer or phone. In the human anatomy lab, they can scan a QR code — those squiggly squares you scan in a restaurant for a menu — on a cadaver theyтАЩre studying to get more information about what theyтАЩre seeing, including videos of surgeries or relevant lectures.
тАЬThis is really very early days,тАЭ Dorsainville says. тАЬBut when augmented reality is done well, itтАЩs really special.тАЭ
More to Come
Health care tends to lag when it comes to technology adoption, Hopkins says.
There are good reasons for that. Doctors need time to prove that a new technology is safe and benefits them and the people they treat. There are also issues related to costs and bringing new technologies into existing workflows.
Despite the challenges, experts say they expect to see more applications of augmented reality in medicine. But which technologies and uses will prove themselves in the clinic largely remains to be seen. YouтАЩre likely to see rapid changes as technology companies work to develop new uses for augmented reality, inside and outside the clinic.
тАЬItтАЩs a big battlefront in terms of who will write the future on this type of technology,тАЭ Dorsainville says.
