High-quality on-patient medical data visualization in a markerless augmented reality environment

Abstract

To provide on-patient medical data visualization, a medical augmented reality environment must support volume rendering, accurate tracking, real-time performance and high visual quality in the final rendering. Another interesting feature is markerless registration, to solve the intrusiveness introduced by the use of fiducial markers for tracking. In this paper we address the problem of on-patient medical data visualization in a real-time high-quality markerless augmented reality environment. The medical data consists of a volume reconstructed from 3D computed tomography image data. Markerless registration is done by generating a 3D reference model of the region of interest in the patient and tracking it from the depth stream of an RGB-D sensor. From the estimated camera pose, the volumetric medical data and the reference model are combined allowing a visualization of the patient as well as part of his anatomy. To improve the visual perception of the scene, focus+context visualization is used in the augmented reality scene to dynamically define which parts of the medical volume will be visualized in the context of the patient’s image. Moreover, context-preserving volume rendering is employed to dynamically control which parts of the volume will be rendered. The results obtained show that the markerless environment runs in real-time and the techniques applied greatly improve the visual quality of the final rendering.