A semi-automatic markerless augmented reality approach for on-patient volumetric medical data visualization

Abstract

In general, augmented reality (AR) applications use polygonal models to augment a real scene. However, most of the medical applications have volumetric data to be visualized. Therefore, it is desirable to the medical AR systems to provide the visualization of these volumes into the real scene in real-time. In this paper we introduce a real-time semi-automatic approach for on-patient medical volume data visualization. It is proposed in a marker less augmented reality (MAR) environment and the medical data consists of a volume reconstructed from 3D computed tomography (CT) image data. A 3D reference model of the region of interest in the patient is generated and tracked from the Kinect depth stream. From the estimated camera pose, a volumetric medical data can be displayed to a physician inside the patient's anatomy at the location of the real anatomy. We evaluate the use of the named standard volume rendering techniques in the context of a MAR environment. The results obtained so far show that these techniques can be applied in this scenario in real-time.