From Liron Pantanowitz and colleagues: Augmented Reality Technology Using Microsoft HoloLens in Anatomic Pathology

Matthew G. Hanna, Ishtiaque Ahmed, Jeffrey Nine, Shyam Prajapati, and Liron Pantanowitz (2018) Augmented Reality Technology Using Microsoft HoloLens in Anatomic Pathology. Archives of Pathology & Laboratory Medicine: May 2018, Vol. 142, No. 5, pp. 638-644.

ORIGINAL ARTICLES

Augmented Reality Technology Using Microsoft HoloLens in Anatomic Pathology

Matthew G. Hanna, MD; Ishtiaque Ahmed, BS; Jeffrey Nine, MD; Shyam Prajapati, DO; Liron Pantanowitz, MD

From the Department of Pathology, University of Pittsburgh Medical Center (UPMC), Pittsburgh, Pennsylvania (Drs Hanna, Nine, and Pantanowitz and Mr Ahmed); Icahn School of Medicine at Mount Sinai, The Mount Sinai Hospital, New York, New York (Dr Prajapati).

The authors have no relevant financial interest in the products or companies described in this article.

Corresponding author: Liron Pantanowitz, MD, Department of Pathology, University of Pittsburgh Medical Center, Shadyside Hospital, Suite 201, 5150 Centre Ave, Pittsburgh, PA 15232 (email: pantanowitzl@upmc.edu).

Context Augmented reality (AR) devices such as the Microsoft HoloLens have not been well used in the medical field.

Objective To test the HoloLens for clinical and nonclinical applications in pathology.

Design A Microsoft HoloLens was tested for virtual annotation during autopsy, viewing 3D gross and microscopic pathology specimens, navigating whole slide images, telepathology, as well as real-time pathology-radiology correlation.

Results Pathology residents performing an autopsy wearing the HoloLens were remotely instructed with real-time diagrams, annotations, and voice instruction. 3D-scanned gross pathology specimens could be viewed as holograms and easily manipulated. Telepathology was supported during gross examination and at the time of intraoperative consultation, allowing users to remotely access a pathologist for guidance and to virtually annotate areas of interest on specimens in real-time. The HoloLens permitted radiographs to be coregistered on gross specimens and thereby enhanced locating important pathologic findings. The HoloLens also allowed easy viewing and navigation of whole slide images, using an AR workstation, including multiple coregistered tissue sections facilitating volumetric pathology evaluation.

Conclusions The HoloLens is a novel AR tool with multiple clinical and nonclinical applications in pathology. The device was comfortable to wear, easy to use, provided sufficient computing power, and supported high-resolution imaging. It was useful for autopsy, gross and microscopic examination, and ideally suited for digital pathology. Unique applications include remote supervision and annotation, 3D image viewing and manipulation, telepathology in a mixed-reality environment, and real-time pathology-radiology correlation.