Range image registration based on 2D synthetic images
Introduction
A 3D scanner is a device to create a point cloud data by scanning a real-world object. The collected point clouds can then be used to generate a 3D digital model in a process called reconstruction. Although there is a variety of technologies available for acquiring the shape of a real-world object, Structured Light Systems (SLS) are currently most popular due to their fast measuring speed, simple optical arrangement, non-contact measurement, moderate accuracy, low cost, and ability to function under varying ambient light conditions [1], [2], [3], [4], [5]. Fig. 1 shows a general configuration of an SLS consisting of a projector and cameras. The projector illuminates the object with a structured light source, and the cameras capture the scene to produce 3D range image. The range image can be considered as a point cloud containing the spatial coordinates for the surface points of the object.
Since a SLS can measure only the area visible from a specific scanning orientation, the full reconstruction of a real-world object requires three major steps: (1) acquiring multiple range images from different viewpoints to cover the entire surface area of the given object, (2) applying a 3D registration process to the acquired range images, and (3) merging the multiple range images to generate a 3D digital model. This paper is focusing on the second step, the registration of range images, of which the accuracy is very important to ensure the quality of the reconstructed 3D model.
The most common method for 3D registration is the well-known Iterative Closest Point (ICP) algorithm [6], which minimizes the distance between two point clouds. The ICP algorithm works for a pair of point clouds, a target and a source. The target point cloud is kept fixed, while the source point cloud is transformed (with a combination of translation and rotation) to minimize the distance between them.
The original ICP algorithm [6] sets the closest points as the correspondence and optimizes the root mean squared deviations. Chen [7] employed point-plane distance by estimating the normal vectors of the point clouds. Ever since then, there have been various researches on the ICP algorithm [8], and they can be roughly categorized into five groups: improving computing speed through efficient correspondence detection [9], [10], [11], [12], [13], [14], improving accuracy by defining the appropriate energy function [15], [16], [17], [18], [19], [20], combining existing algorithms [21], [22], [23], finding correspondence by using probability [19], [24], [25], and eliminating correspondence outliers [26], [27].
Although the ICP algorithm usually offers good results, it has the risk of becoming stuck in a local minimum, especially in cases where there is only a rough initial alignment between the target and the source point clouds. As shown in Fig. 2 (a), the ICP algorithm works well when the two point clouds are within an acceptable range in terms of their position and orientation. The ICP algorithm can become stuck in a local minimum, however, if the two point clouds are outside this acceptable range, as shown in Fig. 2 (b).
To overcome this problem, many researchers have investigated methods to achieve the fine initial alignment of the two point clouds [28]. Most have focused on the use of 3D features in the given point clouds. Previous research on initial alignment can be classified by the type of 3D feature used: curves [29], [30], feature histograms (FPFH) [31], 4-point congruent sets [32], and spin images [33]. Nevertheless, these initial alignment methods using 3D features may not generate useful results when the scanned data has occlusions or a consistent curved geometry such as a cylinder or a cube.
As the relationship between a 2D image and a 3D point cloud can be obtained, the matching relations of 2D features can be used to achieve the initial alignment. Since 2D image registration methods [34], [35] use only the texture and the intensity of the image for calculations, they are much faster than 3D registration and can find 2D features robustly. From this reason, some authors proposed the method using 2D images in 3D registration [36], [37]. However, these methods perform poorly if the viewpoint of the camera moves beyond the threshold degree, because the scene, the geometry, and the intensity of the image changes. Moreover, the algorithm can only perform on a flat and textured surface. This is one of the fundamental problems of 2D image registration algorithms. Therefore, there always exists a risk that registration may return incorrect matching relations, and currently no solution has been found that can eliminate this problem completely.
The objective of this paper is to develop a new registration procedure that minimizes the risk faced by the original ICP algorithm, that of becoming stuck in a local minimum. To reduce this risk, it is necessary to achieve the fine initial alignment of the range images. Our approach is to reduce the risk of using 2D registration by generating multiple pairs of synthetic images from the given range images by changing their resolution and orientation in order to obtain a large number of test cases. Each pair contains two synthetic images, one from the source range image and the other from the target range image. For each pair of 2D synthetic images, we can obtain matching relations using the 2D registration. The proposed approach selects the matching relations which obtain the most votes from the multiple pairs of 2D synthetic images.
As shown in Fig. 3, the proposed registration procedure consists of five steps: (1) synthetizing a 2D image set from both the target range image and the source range image, (2) applying 2D registration to the test cases, each of which is a pair of synthetic images, one from the target image set and the other from the source image set, (3) finding the matching relations that obtain the most votes from the test cases and removing the outliers, (4) transforming the source range image to achieve fine initial alignment, and(5) applying the ICP algorithm to the aligned range images.
The remainder of this paper is organized as follows. Section 2 addresses the proposed approach to the fine initial alignment of given range images with image registration. Section 3 describes the details and the overall procedure of the proposed algorithm, and Section 4 shows the results of the experimentation. Finally, concluding remarks are given in Section 5.
Section snippets
Approach to the fine initial alignment of range images with image registration
As mentioned earlier, initial alignment algorithms work on two range images (a target point cloud and a source point cloud). Since a range image is generated from 2D images from an SLS, the correspondence between a 2D image pixel and a 3D point cloud vertex can be obtained. Therefore, 2D image registration can be used to achieve the initial alignment of the 3D data. The 2D image registration algorithm works on two given images (a source image and a target image), and consists of two major
Proposed 3D registration algorithm
To generate the synthetic images from the different orientations, the back-projection method is used. As the calibration information of the camera is known, the correspondence between the point cloud vertex and the image pixels can be obtained from back-projection. In the back-projection of the 3D point cloud to the image plane, formulas (1), (2) are used. where and are the coordinates of the image plane (pixels), and , and are the Cartesian coordinates of the
Experiment set-up
The proposed algorithm was implemented in C++ language and test runs were made on a personal computer with an i5-2500 processor with 8 GB memory and a Windows 7 operating system. For the experimentation, we used the commercial scanner REXCAN4® developed by Medit. As shown in Fig. 11, three models were used for the experiments: a teeth model, a dinosaur model, and a mechanical part.
In the experiment, we compared the proposed algorithm with the existing algorithms FPFH [31], ICPIF [29], SURF [34]
Summary
In this research, a new registration procedure for the range images acquired by an optical 3D scanner is proposed. Since registration has a significant impact on the quality of a reconstructed CAD model, it is essential to ensure the stability of the registration algorithm. One of the major attributes of the proposed procedure is that it minimizes the major risk of the original ICP algorithm, becoming stuck at a local minimum, by achieving the fine initial alignment of the range images. For the
Acknowledgments
This research was supported by the Technology Innovation Program (10065150, Development of Low-cost and Small LIDAR System Technology based on 3D Laser scanning for 360 Real-time Monitoring), funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea) and the Korea Evaluation Institute of Industrial Technology (KEIT, Korea) . This work was also partially supported by National Research Foundation (NRF-2015R1A2A2A01005871) by the Ministry of Education, Science and Technology, Korea.
References (45)
- et al.
Automated scanning of dental impressions
Comput Aided Des
(2009) - et al.
Reverse engineering with a structured light system
Comput Ind Eng
(2009) - et al.
Next viewing directions for the scanning of dental impressions
Comput Aided Des
(2015) - et al.
Simultaneous registration of multiple views with markers
Comput Aided Des
(2009) - et al.
Object modeling by registration of multiple range views
Image Vis Comput
(1992) - et al.
An accurate and fast point-to-plane registration technique
Pattern Recognit Lett
(2003) - et al.
Fully automatic registration of multiple 3D data sets
Image Vis Comput
(2003) Automatic 3D free form shape matching using the graduated assignment algorithm
Pattern Recognit
(2005)- et al.
Newton methods for parametric surface registration, Part I. Theory
Comput Aided Des
(2003) - et al.
Registration without ICP
Comput Vis Image Underst
(2004)
Extracting lines of curvature from noisy point clouds
Comput Aided Des
Image registration methods: A survey
Image Vis Comput
SURF:Speeded up robust features
Comput Vis Image Underst
Reverse engineering of a symmetric object
Comput Ind Eng
A method for registration of 3D shapes
IEEE Trans Pattern Anal Mach Intell
Efficient variants of the ICP algorithm
Fast global registration of 3D sampled surfaces using a multi-z-buffer technique
Image Vis Comput
Real-time 3D registration using GPU
Mach Vis Appl
A scale stretch method based on ICP for 3D data registration
IEEE Trans Autom Sci Eng
Cited by (23)
Level-Set-based Wheel–Rail Profile Evolution in Four-Dimensional Continuous Functional Space
2023, CAD Computer Aided DesignPoint cloud registration and localization based on voxel plane features
2022, ISPRS Journal of Photogrammetry and Remote SensingCitation Excerpt :Except for three-dimensional coordinate value, other helpful information for registration can be obtained: the point cloud may also include color or intensity information (where the intensity information can be regarded as a grayscale image). Here, the image method can be used to construct texture features, such as SIFT (Zhang et al., 2019), SURF (Jung et al., 2018), ORB (Ji et al., 2013) and other features in the image algorithm. In addition to these features, point cloud features based on learning have been greatly developed in recent years, providing the possibility to obtain more advanced features.
Part-in-whole point cloud registration for aircraft partial scan automated localization
2021, CAD Computer Aided DesignCitation Excerpt :Weighted ICP registration. ICP [22] and its variants [23–28] are a set of fine registration algorithm based on the Iterative Closest Point scheme which tries to minimize the difference between two clouds of points iteratively. The most important step among ICP is to find the correct correspondences between two point clouds.
High-precision detection method for large and complex steel structures based on global registration algorithm and automatic point cloud generation
2021, Measurement: Journal of the International Measurement ConfederationCitation Excerpt :At present, the most common method to realize multi-station registration is ICP algorithm. Jung et al. proposed a new procedure of range image (point cloud) registration reducing the risk of the original Iterative Closest Point (ICP) algorithm which gets stuck in a local minimum [18]. On the basis of rough registration, He et al. applied the theory of least squares indirect adjustment to rapidly register point clouds through the nearest point iteration algorithm [19].
A fiducial tag invariant to rotation, translation, and perspective transformations
2018, Pattern RecognitionCitation Excerpt :At least four points are needed to estimate a homography between a tag’s squared border and the corresponding tag seen in the image that allows its relative orientation and translation estimation. The matching problem has been widely studied for general images [25–27] with many points (dozens or hundreds), and for 3D point clouds [28], but for the visual markers, most of approaches with squared border solve the matching using only the four corners. In this case the ambiguity introduced by the symmetry at 0, 90, 180, and 270 degrees is solved by additional mechanisms after the homography is estimated.