Developing structural constraints for accurate registration of overlapping range images

https://doi.org/10.1016/j.robot.2004.02.002Get rights and content

Abstract

Automatic image registration is an attractive and unresolved problem in the machine vision literature. This paper presents two novel structural constraints, namely proximity and closeness constraints, that improve both the accuracy and robustness of an existing motion consistency based algorithm (GICP) for automatic image registration. It also defines the conditions when such constraints are fired at specific points of the registration of two overlapping range images. The proximity constraint says that neighbouring points should be neighbouring before and after a rigid motion. The closeness constraint implements a local closest point search from the second image to the first image. While the GICP algorithm uses an exhaustive search for the closest points, in this paper, we employ the optimised k–d tree data structure to accelerate the closest point search. A large number of experiments based on real range images demonstrate that the combination of rigid motion constraints with these novel proximity and closeness constraints leads to a more accurate and robust evaluation of possible correspondences leading, in turn, to more accurate, robust and efficient automatic image registration results.

Introduction

New technological developments in optics and electronics are rendering laser scanning systems cheaper and more accurate. Such systems can directly capture depth information from objects, simplifying range image analysis and enlarging their applications’ scope. Range images consist of sets of 3D structural data points and describe free form surfaces of objects from different viewpoints. The registration of range images is an important problem widely acknowledged in the machine vision literature. Many methods have been proposed to tackle the registration problem, such as techniques based on the scatter matrix [13], geometric histogram [1], iterative closest point (ICP) [2], [4], [30], graph matching [6], extremal points [23], distance based search [3], and interactive method [18] among many others. These registration algorithms often have a common characteristic that they can succeed in one situation, but also can degenerate catastrophically in another. Hence, automatic image registration still remains an open problem. Registration techniques find applications in many areas such as, for instance, object recognition, motion estimation, scene understanding, and robot autonomous navigation [10].

Among these methods, the ICP algorithm has attracted much attention from the machine vision community since it was proposed in 1992 independently by several researchers. However, the ICP algorithm assumes that a good initialisation of motion parameters is available and unavoidably introduces false matches in almost every iteration due to occlusion and appearance and disappearance of points, as it uses only a distance constraint to determine correspondents in 3D space and one constraint cannot uniquely define a 3D point (the ICP algorithm assumes that the correspondent of a point in the first image is the nearest point in the second image to its transformed point). Thus, the key to successfully applying the ICP algorithm to establish correspondences between the images to be registered is to eliminate false point matches. Addressing this limitation, a large number of improvements to the traditional ICP algorithm have been proposed. The main techniques applied are based on discarding boundary points [24], thresholding false matches [11], [12], matching interpoint distances [8], matching bitangent curves [26], matching surface signatures [28], matching spin images [10], extracting and matching space curves (curvature and torsion) [29], checking interpoint distance consistency [7], checking orientation consistency [19], [30], or increasing the dimensionality of points by incorporating invariants [21], [22], colour [11], or normal vector [8] among others.

Analyses of the improvement methods have revealed that all steps of the traditional ICP algorithm are refined and the improvements are mainly based on matching a number of structural features described in a single coordinate frame. Unfortunately, an accurate extraction of these features is also a challenging task in the machine vision community. Addressing this issue in [14], [15], [17], [20], we developed rigid motion constraints, bridging the points described in two different coordinate frames before and after a rigid motion. These constraints were successfully used to deal with false point matches, thus leading to more accurate image registration and motion estimation results. However, the algorithm we proposed, geometric ICP (GICP), was designed for the automatic registration of sets of unorganised points and did not make use of any geometric information existing between neighbouring points. As a result, the GICP algorithm sometimes does converge to a local minimum leading to inaccurate image registration and motion estimation results. We seek to combine the advantage of GICP with those of using structured data. It is acknowledged that, in general, algorithms based on structured data yield better registration results [5].

A preliminary version of this paper appears in [16]. In this paper we extend the GICP algorithm as described in [14] to structured data by developing two novel constraints, namely proximity and closeness constraints. The consequence of incorporating these constraints into the GICP algorithm is that more refined (more accurate) sets of correspondences are obtained and this leads to more accurate and robust automatic image registration and motion estimation results. While the GICP algorithm uses an exhaustive search for the closest points, in this paper, we employ the optimised k–d tree data structure [9] to accelerate the closest point search so that the improved GICP algorithm is also computationally efficient.

The proximity constraint says that neighbouring points in one image should remain neighbouring points in the transformed image. The closeness constraint considers distances between neighbouring points described in two different images and implements a local closest point search from the second image to the first image. These two constraints do not require any feature extraction or image segmentation and thus, avoid the difficulties in extracting point based structural features [10], [26], [29] for matching or segmenting images to extract high order (e.g., planar [27]) features.

In order to provide a better understanding of the proposed algorithm for automatic image registration, we also implemented the original GICP algorithm as proposed in [14]. A large number of experiments based on real images captured under typical imaging conditions have indeed demonstrated that the combination of rigid motion constraints with the proximity and closeness constraints leads to more effective, accurate and robust evaluation of possible correspondences and thus, to more accurate, robust and efficient image registration and motion estimation results.

The rest of this paper is organised as follows. Section 2 outlines the major steps in the GICP algorithm, Section 3 develops the novel structural constraints, Section 4 extends the GICP algorithm to structured image points data, and Section 5 presents experimental results. Finally, Section 6 discusses a number of issues affecting the performance of registration algorithms and draws some conclusions.

Section snippets

Outline of the GICP algorithm

In [14], [20] the geometric properties of reflected correspondence vectors were analysed and translated into global rigid motion constraints bridging the points described in different coordinate frames before and after a rigid motion. The constraints were successfully used to develop the GICP algorithm, whose three major steps are outlined as follows:

  • 1.

    Use the traditional ICP criterion [2], [4], [30] to establish a set of possible correspondences between the two images to be registered.

  • 2.

    Use the

Developing structural constraints

Assume that each range image file consists of four parts: flag, x coordinate, y coordinate, and z coordinate. The flag part f(i,j) specifies whether the point in the first image at row i and column j is valid or not. Each part is of size m×n. In order to speed up the search for the closest points, we extract all valid points pi (i=1,2,…,n1) and pi′(i=1,2,…,n2) from the image files in a sequential manner. Due to occlusion and appearance and disappearance of points, n1 here is not necessarily

The novel algorithm

Given two range images P and P′ to be registered, for each valid point pi (i=1,2,…,n1) in P, if one of its non-diagonal neighbouring points is invalid, then we label it as bi=−1. Otherwise, we label it as bi=1. Similarly, for each valid point pi′(i=1,2,…,n2) in P′, if one of its non-diagonal neighbouring points is invalid, then we label it as bi′=−1. Otherwise, we label it as bi′=1. We set the maximum iteration number as M, the threshold ρ specifying the desired registration error, and the

Experimental results

In order to obtain a better understanding of the performance of the proposed algorithm, the original GICP algorithm and the proposed SICP algorithm were all implemented on a Pentium III, 500 MHz, 256 MB RAM computer. The initialisation parameters were used as described in [14] for both the GICP and SICP algorithms where the quaternion q(0)=(99/10,0.1,0,0)T, representing rotation matrix R, and translation vector t(0)=p̄′−p̄ and p̄′ and p̄ are centroids of the second and first image points. We

Discussion

Through a large number of experiments based on real images, the following observations have been made.

The quality of images plays a crucial role for accurate automatic registration of such images [15], [20]. For high quality images, different algorithms often exhibit similar evolutionary behaviour for registration. For low quality images, all algorithms have difficulty in registering them and they often exhibit different evolutionary behaviours for registration. Thus, it will be useful to

Acknowledgements

We would like to express our sincere thanks to Dr Lynda Thomas from the University of Wales at Aberystwyth for her comments that have improved the readability of the paper. We would also like to express our appreciation to the anonymous reviewers, editor, and Professor Ross King from the University of Wales at Aberystwyth for their valuable comments that have improved the quality of the paper.

Yonghuai Liu started in September 2001 as a Lecturer at the University of Wales, Aberystwyth. He obtained his first Ph.D. degree in 1998 from Northwestern Polytechnical University, PR China, and second Ph.D. degree in 2001 from The University of Hull, United Kingdom. From April to September 1999, he worked as a research assistant at The University of Hull. From March 2000 to September 2001, he worked as a research fellow at Sheffield Hallam University. He has published more than 60 papers in

References (30)

  • J.H. Friedman et al.

    An algorithm for finding best matches in logarithmic expected time

    ACM Transactions on Mathematical Software

    (1977)
  • D. Huber, O. Carmichael, M. Hebert, 3D map reconstruction from range data, in: Proceedings of the ICRA, 2000, pp....
  • A. Johnson, S.B. Kang, Registration and integration of textured 3D data, Technical Report, CRL 96/4 DEC, Cambridge...
  • Z. Lin, H. Lee, T. Huang, Finding 3D point correspondences in motion estimation, in: Proceedings of the ICPR, 1986, pp....
  • Y. Liu, M.A. Rodrigues, Y. Wang, Developing rigid motion constraints for the registration of free-form shapes, in:...

    • Cited by (0)

    Yonghuai Liu started in September 2001 as a Lecturer at the University of Wales, Aberystwyth. He obtained his first Ph.D. degree in 1998 from Northwestern Polytechnical University, PR China, and second Ph.D. degree in 2001 from The University of Hull, United Kingdom. From April to September 1999, he worked as a research assistant at The University of Hull. From March 2000 to September 2001, he worked as a research fellow at Sheffield Hallam University. He has published more than 60 papers in international conference proceedings and journals. He is a Guest Editor for the special issue of Computer Vision and Image Understanding Journal on the Registration and Fusion of Range Images published in 2002. His primary research interests lie in machine vision, image registration, motion estimation, pattern recognition, image processing, machine learning, computer graphics, artificial intelligence, and intelligent systems.

    Baogang Wei received his Ph.D. degree in computer science from Northwestern Polytechnical University, PR China, in 1997. From September 1997 to August 1999, He worked as a Post-doctor at Zhejiang University. He is now an Associate Professor in the College of Computer Science and Technology at the University. He has published more than 30 papers in international conference proceedings and journals. His research interests include artificial intelligence, pattern recognition, image processing, computer graphics, and database and knowledge base systems.

    View full text
    Copyright © 2004 Elsevier B.V. All rights reserved.