Multi-frame adaptive non-rigid registration for markerless augmented reality

Abstract

Augmented Reality is a technology in which the user's view of a real scene is augmented with virtual information. To provide real-time performance, the major of its applications only support rigid interaction with the fiducial marker (i.e. marker-based augmented reality) or the part of the real scene being used as a natural marker (i.e. markerless augmented reality). In this context, when the natural marker consists of a deformable object (e.g. face, body, hand), it is desirable for the application to support non-rigid interactions between the user and the marker. In this paper we present an adaptive non-rigid surface registration algorithm for markerless augmented reality. It is applied in a multi-frame manner to achieve fast performance. Likewise, it takes advantage from the power of the graphics processing unit to improve application's performance. Based on multi-frame adaptivity, we show that the markerless tracking runs almost in real-time, improving the accuracy of the tracking when compared to the rigid-only solution.