Bayesian perspective for the registration of multiple 3D views☆
Introduction
Current technology allows to obtain 3D representations of real objects or scenarios by using different methods. From usual methods like stereoscopy to complex devices like LADAR or time-of-flight cameras, the possibilities are growing continuously. However, acquisition techniques usually have problems with occluded surfaces or the limited field of view, so it is usually necessary to combine different views of the same object or scenario in order to obtain a full representation. Using this process another problem then arises: the registration of these individual 3D views which will enable, at the end of the process, to a whole 3D reconstruction of the desired object or scene.
First step for this objective is the so-called pairwise registration, where the 3D representations are registered pair to pair. Depending if the two structures have overlap, pairwise registration methods will give as a result a transformation which registers the first 3D view to the second one.
Multiview registration is the second step of this process, and is usually a more complex situation. Assuming that pairwise registrations are correct, their concatenation along a cycle will probably result in a non-satisfactory multiview registration because of the accumulation of the different pairwise errors. An example of this effect can be seen in Fig. 1.
In addition, there could exist another situation which produces more problems. Even if two structures register perfectly in the pairwise registration process, their transformation could be incorrect in a global environment. This could happen specially if we are working with objects with symmetries, planes or repetitive patterns. In these cases, most of the current multiview registration algorithms will fail because they are not ready to deal with this kind of error.
The main contribution of this paper, in opposition to other state-of-the-art papers, is the possibility of detecting these incorrect registrations between different views and therefore minimize their impact in the global registration process. This feature is achieved thanks to the use of different weights which encode the reliability we have in the correspondences between the views.
The structure of the document is as follows: a review of different registration algorithms is presented in Section 2, followed by an introduction to the main problematic of multiview registration in Section 3. The proposed algorithm is explained in detail in Section 4, and the obtained experimental results are shown in Section 5. Finally, conclusions and possible future improvements are explained at the end of the paper in Section 6.
Section snippets
State of the art
As previously stated the first step in order to achieve the registration of multiple 3D representations is the computation of the pairwise registrations. The most usual method consists in the establishment of correspondences between points of the two views, which can be found by using descriptors like the Spin-Images of Johnson and Hebert [1] or covariance descriptors like the one presented by Fehr et al. in [2]. The combination of correspondences between the two views determines an
Problematic issues in the multiview registration process
The most usual problem related with the multiview registration process is the minimization of the global error which is produced due to the small errors in every pairwise registration. These errors could be produced by different factors, like a noisy acquisition process, differences in the 3D structure of two overlapping views due to the different date of acquisition, or, in most of the cases, by an inaccurate selection of correspondences. Having a look at the example already shown in the left
Bayesian-based multiview registration method
In this section our developed method for the 3D registration of multiple views is explained, which will be called Bayesian-Based Multiview Registration (BBMR) method in the following. As previously stated, this method is based on the previous work presented by Krishnan et al. in [13]. In our work we apply an additional layer over this method, adding an uncertainty which is solved thanks to the use of the Expectation Maximization algorithm.
As an introduction to the problem, we first describe the
Experimental results
In this paper we focus on two main issues: the uncertainty on pairwise correspondences and the combination of global and local information in the registration process of multiple views.
Our technique needs, as a starting point, setting up correspondences between pairs of points belonging to two different overlapping views. These pairwise correspondences may contain some errors due to the fact that they are established by using only the information from the corresponding pair of views, which in
Conclusions and future work
This paper presents our Bayesian-Based Multiview Registration (BBMR) method for the registration of multiple 3D scans. The main property of our BBMR method consists in the property of being tolerant to a certain number of incorrect correspondences which could be caused by different factors like an incorrect manual selection, symmetries on the scanned 3D object or repetitive patterns. This tolerance is achieved thanks to the use of an additional layer placed over an existing multiview
Acknowledgment
This work was partially funded by the research project TIN2012-39203.
References (27)
- et al.
Object modeling by registration of multiple range images
Image Vis. Comput.
(1992) - et al.
Registration of surfaces minimizing error propagation for a one-shot multi-slit hand-held scanner
Pattern Recog.
(2008) - et al.
Simultaneous registration of multiple range views for use in reverse engineering of CAD models
Comput. Vis. Image Understand.
(1998) - et al.
Multiview range image registration using the surface interpenetration measure
Image Vis. Comput.
(2007) - et al.
Fully automatic registration of multiple 3D data sets
Image Vis. Comput.
(2003) - et al.
Using spin images for efficient object recognition in cluttered 3D scenes
IEEE Trans. Pattern Anal. Mach. Intell.
(1999) - D. Fehr, A. Cherian, R. Sivalingam, S. Nickolay, V. Morellas, N. Papanikolopoulos, Compact covariance descriptors in 3D...
- et al.
A method for registration of 3-D shapes
IEEE Trans. Pattern Anal. Mach. Intell.
(1992) - K. Pulli, Multiview registration for large data sets, in: Proceedings of the International Conference on 3-D Digital...
- A. Nuchter, H. Surmann, K. Lingemann, J. Hertzberg, S. Thrun, 6D SLAM with an application in autonomous mine mapping,...
Multiview registration of 3D scenes by minimizing error between coordinate frames
IEEE Trans. Pattern Anal. Mach. Intell.
An efficient and accurate method for the relaxation of multiview registration error
IEEE Trans. Image Process.
Cited by (29)
Effective multiview registration of point clouds based on Student's-t mixture model
2022, Information SciencesCitation Excerpt :However, if objects are symmetric, then the transformation of two views may be incorrect in terms of the global transformation, even if they are registered correctly during pairwise registration. Accordingly, Mateo et al. [27] introduced a Bayesian framework to take advantage of different weights that represent the reliability of the correspondences among different views, so that incorrect registrations can be detected. Although this method minimizes the impact of incorrect global pairwise registrations, its efficiency is poor as it requires the estimation of a huge number of variables, and the total number of registration errors are too large, despite avoiding sequential registrations.
Least-squares registration of point sets over SE(d) using closed-form projections
2019, Computer Vision and Image UnderstandingMultiple point sets registration based on Expectation Maximization algorithm
2018, Computers and Electrical EngineeringCitation Excerpt :In the multi-view registration problem, Govindu and Pooja [12] applied the motion average algorithm to tackle it. Mateo et al. [13] casted the problem into the Bayesian framework. However, current multi-view registration algorithms always fail in achieving satisfactory results, due to the accumulation of the different pairwise errors and limitations of methods.
Local to global registration of multi-view range scans using spanning tree
2017, Computers and Electrical EngineeringCitation Excerpt :By viewing scans as realizations of a Gaussian mixture (GMM), Georgios et al. [19] casted the registration problem into a clustering problem, which can be solved by the Expectation-Maximization (EM) algorithm. Mateo et al. [20] proposed a multi-view registration approach within the Bayesian framework. By considering pairwise correspondences as missing data, multi-view registration can be viewed as the inferred problem of maximizing a posteriori, which can also be solved by using the EM algorithm.
Automatic multi-view registration of unordered range scans without feature extraction
2016, NeurocomputingCitation Excerpt :Since they do not update the correspondences through registration process, the multi-view registration errors cannot actually be reduced by these approaches. Recently, Mateo [17] casted the multi-view registration problem into the Bayesian framework, which treats pair-wise correspondences as missing data by an Expectation-Maximization algorithm. The related work also appears in [18].
- ☆
This paper has been recommended for acceptance by K.L. Boyer