Accurate non-rigid registration based on heuristic tree for registering point sets with large deformation
Introduction
In medical image processing and pattern recognition, such as medical image segmentation [1], [2], and image retrieval and classification [3], [4], registration based on the feature points is the foundation, especially the registration of point sets representing position. The task of registration is to find the best correspondence between the subject and the model point sets, and yield the optimal alignment with the good spatial transformation. The effective and simple method is the iterative closest point (ICP) algorithm [5], [6], [7]. Moreover, a great number of scholars devote great efforts to enhancing the performance of ICP on the robustness and speed. For instance, to improve the robustness, the improvement of robustness was realized [8] by introducing a new hybrid genetic algorithm technique and evaluation metric based on surface interpenetration, and a probability ICP algorithm was proposed for rigid registration of point sets with noise [9]. Meanwhile, Kim et al. [10] improved the speed with two acceleration techniques which were hierarchical model point selection and logarithmic data point search. A combination of ICP variants which assumed a good initial guess, was proposed to optimize for high speed [11].
For the purpose of improving the performance of the ICP, many scholars make great efforts to extend the applications for the scaling case. An isotropic scaling ICP was proposed by Ying et al. [12] through integrating a scale factor, and the registration was formulated into a constraint optimization problem. As the above-mentioned method could not handle the case with different scales, Du et al. further presented the anisotropic scaling ICP algorithm which incorporated a scale matrix with boundaries into the original ICP for scaling registration [13]. Moreover, to deal with the affine case, based on lie group, the affine registration problem was ultimately simplified to a quadratic programming problem [14]. To enhance the stability, Zhu et al. [15] proposed the affine ICP algorithm with bidirectional distance to solve the affine registration problem. Meanwhile, Amberg et al. [16] presented to employ a locally affine regularization which assigned an affine transformation to each vertex and minimized the difference of neighboring vertices transformation. The above-mentioned methods about scaling and affine registration based on ICP have high precision and fast speed, but the above registration approaches greatly depending on the initial values are easily trapped into local minima and not robust enough.
As a consequence, the approaches based on the softassign method emerged. For instance, the thin plate spline-robust point matching (TPS-RPM) algorithm used TPS as the parameterization of the non-rigid spatial mapping and the softassign for the correspondence [17], but the searching speed was very slow. Meanwhile, in the non-rigid case, the coherent point drift (CPD) [18] imposed the coherence constraint by regularizing the displacement field and using the variational calculus to derive the optimal transformation. Moreover, some scholars extended to graph matching methods, and applied them to correspondence detection of general images [19], [20]. However, the above-mentioned methods are effective for the images that have similar structures or appearances, but when the images bear large deformations, the registration doesn’t accomplish satisfactory results.
Meanwhile, there are approaches to deal with the large deformation problem. Jia et al. [21] proposed the tree-based approach to handle this case, but it employed the mean squared difference of the intensity differences and was used primarily for image segmentation which didn’t meet our requirement of point set registration. Moreover, Ying et al. [22] proposed to use intensity differences to build the graph, and then apply a groupwise nonlinear registration method for image registration. Different from these methods, we propose the tree-based registration method based on the novel shape similarity to realize the optimal alignment of point sets with large differences. Moreover, the shape similarity is measured by the affine ICP algorithm with bidirectional distance which could avoid the ill-posed phenomenon that the affine transformation is close to zero. And then the shape similarity is used to build the tree with the nodes standing for model or subject images and the edges connecting model/subject image pairs with small shape differences. Point sets with small structural discrepancies are added to the tree heuristically until all subjects are constructed in the tree. By making full use of the information from image distribution to progressively conduct the non-rigid registration, the subjects with large differences are accurately registered by the model. We comprehensively evaluate the proposed framework associated with the state-of-the-art non-rigid registration methods including TPS and CPD on the part B of CE-Shape-1 [23]. Consequently, it achieves significant improvement compared with the traditional TPS and CPD individually, in terms of accuracy.
This paper is organized as follows. In Section 2, the typical non-rigid algorithms TPS and CPD are briefly introduced. Following that is Section 3, where the specific process of the proposed framework is analyzed. Next, the framework combined with TPS and CPD is verified advantageously to the point sets of large deformations via the experiments in Section 4. Finally, the conclusion is drawn in Section 5.
Section snippets
The traditional non-rigid registration methods
In practice, the most conventional non-rigid registration methods including the TPS and CPD algorithms adopt softassign method based on the full probability. Among them, the TPS algorithm uses deterministic annealing and alternate updates for soft assignment and parameters estimation [17]. Meanwhile, the CPD algorithm regularizes the displacement field between two point sets following the motion coherence theory [18]. Next, we briefly introduce the two methods.
The proposed framework
As the database contains a considerable number of individual subjects for the same object, there is no defying the fact that some subjects bear large structural discrepancies with respect to the model. Therefore, it is a big challenge to accomplish the non-rigid registration of point sets with large deformations. For the purpose of handling this case, the large deformation can be progressively segmented into several small discrepancies through the heuristic tree. In this tree, the images with
Experimental results
In this part, we evaluate the proposed framework on the part B of CE-Shape-1[23]. As the TPS [17] and CPD [18] algorithms are the traditional non-rigid registration methods, the proposed framework is combined with TPS and CPD, and then they are named Tree-TPS and Tree-CPD, respectively, which are compared with the traditional TPS and CPD to demonstrate the effectiveness and the accuracy. We randomly select one image as the model node, and other images as subjects. All the tests are done by
Conclusion
In this paper, we propose the new heuristic tree-based matching framework to deal with the problem of non-rigid registration of point sets with large deformation. For this purpose, we propose to use the affine ICP algorithm to learn the image distribution, and then build the heuristic tree and complete the non-rigid registration. The experiments verify the robustness and precision of the framework compared with the traditional non-rigid registration methods. The main contributions of the
Acknowledgements
This work was supported by 973 Program under Grant no. 2015CB351703, the National Natural Science Foundation of China under Grant no. 91320301, the Natural Science Basic Research Plan in Shaanxi Province of China under Grant nos. 2014JM8336 and 2012JQ8032, and the Program of Introducing Talents of Discipline to University under Grant no. B13043.
Shaoyi Du received B.S. degrees both in computational mathematics and in computer science, M.S. degree in applied mathematics and Ph.D. degree in pattern recognition and intelligence system from Xi’an Jiaotong University, China in 2002, 2005 and 2009, respectively. He worked as a postdoctoral fellow in Xi’an Jiaotong University from 2009 to 2011 and visited University of North Carolina at Chapel Hill from 2013 to 2014. He is currently an associate professor of the Institute of Artificial
References (31)
- et al.
LEAP: learning embeddings for atlas propagation
NeuroImage
(2010) - et al.
Statistical shape models for 3D medical image segmentation: a review
Med. Image Anal.
(2009) - et al.
Object modelling by registration of multiple range images
Image Vision Comput.
(1992) - et al.
Probability iterative closest point algorithm for m-D point set registration with noise
Neurocomputing
(2015) - et al.
Scaling iterative closest point algorithm for registration of m-D point sets
J. Visual Commun. Image Represent.
(2010) - et al.
Affine iterative closest point algorithm for point set registration
Pattern Recognit. Lett.
(2010) - et al.
A new point matching algorithm for non-rigid registration
Comput. Vision Image Understanding
(2003) - et al.
A new graph matching method for point-set correspondence using the EM algorithm and Softassign
Comput. Vision Image Understanding
(2012) - et al.
Smooth point-set registration using neighboring constraints
Pattern Recognit. Lett.
(2012) - et al.
Iterative multi-atlas-based multi-image segmentation with tree-based registration
NeuroImage
(2012)
Hierarchical unbiased graph shrinkage (hugs): a novel groupwise registration for large data set
NeuroImage
Hyperspectral Image Classification Through Bilayer Graph Based Learning
IEEE Trans. Image Process.
Visual-textual Joint Relevance Learning for Tag-based Social Image Search
IEEE Trans. Image Process.
Iterative point matching for registration of free-form curves and surfaces
Int. J. Comput. Vision
Cited by (10)
Anatomical landmark detection on 3D human shapes by hierarchically utilizing multiple shape features
2017, NeurocomputingCitation Excerpt :The key ingredient of the method is that it builds a common template shape in a groupwise way by taking all the training shapes into account. Of note, The registration technique provides an effective way for aligning two/multiple point clouds, and various improvements have been made to enhance its accuracy [22] and robustness [23,24]. However, since a pure point cloud ignores the information how neighboring points are connected, these methods usually face challenges under large shape deformation such as articulation.
Movable type printing identification based on Tangut characters registration
2017, NeurocomputingCitation Excerpt :Even the words printed by the same movable character model may be distorted during the process of printing and the character image acquisition. To solve the problem of the characters image registration of large deformation, we segment the large deformation into several small differences along the tree/graph [14–16]. In a tree/graph structure, the character images with similar shape are connected together, while the others with big differences are linked indirectly through several layers.
Affine scale space: an affine invariant image structure to promote the detection of correspondences from stereo images
2017, NeurocomputingCitation Excerpt :A scale space can be defined as a collection of several pre-smoothed images by different sized Gaussian kernels [4]. In general, scale space theory is a framework for multi-scale signal representation to handle with image structures at different scales, by employing a one-parameterized family of smoothed images [16]. This framework provides a scale-invariant representation [17], which is necessary to deal with variations that may occur in image data [18].
A robust nonrigid point set registration method based on collaborative correspondences
2020, Sensors (Switzerland)Joint Template Matching by Candidate Set Voting
2018, Proceedings 2018 Chinese Automation Congress, CAC 2018
Shaoyi Du received B.S. degrees both in computational mathematics and in computer science, M.S. degree in applied mathematics and Ph.D. degree in pattern recognition and intelligence system from Xi’an Jiaotong University, China in 2002, 2005 and 2009, respectively. He worked as a postdoctoral fellow in Xi’an Jiaotong University from 2009 to 2011 and visited University of North Carolina at Chapel Hill from 2013 to 2014. He is currently an associate professor of the Institute of Artificial Intelligence and Robotics in Xi’an Jiaotong University. His research interests include pattern recognition, machine learning and computer vision.
Juan Liu received Bachelor degree in automation from Xi’an Jiaotong University, China in 2013. She is currently a graduate student in the institute of Artificial Intelligence and Robotics in Xi’an Jiaotong University. Her research interests include mobile robot and image registration.
Chunjia Zhang received Bachelor degree from Nanjing University of Technology, China in 2008. He is currently a Ph.D. Candidate in the institute of Artificial Intelligence and Robotics in Xi’an Jiaotong University. His research interests include mobile robot and image registration.
Meifeng Xu received B.S. degrees in Clinical Medicine from Tongji Medical College of Huazhong University of Science and Technology, China in 2002. She studied Mohs surgery technique, facial reconstruction, laser surgery and other cosmetic procedures as a visiting scholar in Warren Skin Care Center, NJ, USA from 2013 to 2014. She is currently an attending physician in the Department of Dermatology, The Second Affiliated Hospital of Xi’an Jiaotong University. Her research interests include dermatology surgery technique and medical image analysis.
Jianru Xue received the B.S. degree from Xi’an University of Technology in 1994, and received the M.S. and Ph.D. degrees from Xi’an Jiaotong University, Xi’an, China, in 1999 and 2003, respectively. He had worked in FujiXerox, Tokyo, Japan, from 2002 to 2003, and visited University of California, Los Angeles, from 2008 to 2009. He is currently a professor of Institute of Artificial Intelligence and Robotics of Xi’an Jiaotong University. His research field includes computer vision, visual navigation, and video coding based on analysis.