Elsevier

Neurocomputing

Volume 168, 30 November 2015, Pages 681-689
Neurocomputing

Accurate non-rigid registration based on heuristic tree for registering point sets with large deformation

https://doi.org/10.1016/j.neucom.2015.05.056Get rights and content

Abstract

The conventional thin plate spline (TPS) and coherent point drift (CPD) methods are effective to yield non-rigid registration of point sets with similar appearance or structure, but it is a big challenge to register point sets with large deformation. To handle this case, by fully utilizing the heuristics derived from the distribution of entire population, this paper proposes a novel framework for non-rigid registration of point sets with large deformation via the heuristic tree matching. First, we use affine ICP with bidirectional distance to measure the shape similarity between two point sets. Then, the heuristic tree is built based on shape similarities, which connects two point sets with small deformation. In this way, the large deformation is divided into several small differences. Finally, the non-rigid registration is conducted progressively according to the tree. Experimental results demonstrate the proposed framework is valid for the alignment of point sets with large deformation and is more accurate compared with the traditional non-rigid registration approaches.

Introduction

In medical image processing and pattern recognition, such as medical image segmentation [1], [2], and image retrieval and classification [3], [4], registration based on the feature points is the foundation, especially the registration of point sets representing position. The task of registration is to find the best correspondence between the subject and the model point sets, and yield the optimal alignment with the good spatial transformation. The effective and simple method is the iterative closest point (ICP) algorithm [5], [6], [7]. Moreover, a great number of scholars devote great efforts to enhancing the performance of ICP on the robustness and speed. For instance, to improve the robustness, the improvement of robustness was realized [8] by introducing a new hybrid genetic algorithm technique and evaluation metric based on surface interpenetration, and a probability ICP algorithm was proposed for rigid registration of point sets with noise [9]. Meanwhile, Kim et al. [10] improved the speed with two acceleration techniques which were hierarchical model point selection and logarithmic data point search. A combination of ICP variants which assumed a good initial guess, was proposed to optimize for high speed [11].

For the purpose of improving the performance of the ICP, many scholars make great efforts to extend the applications for the scaling case. An isotropic scaling ICP was proposed by Ying et al. [12] through integrating a scale factor, and the registration was formulated into a constraint optimization problem. As the above-mentioned method could not handle the case with different scales, Du et al. further presented the anisotropic scaling ICP algorithm which incorporated a scale matrix with boundaries into the original ICP for scaling registration [13]. Moreover, to deal with the affine case, based on lie group, the affine registration problem was ultimately simplified to a quadratic programming problem [14]. To enhance the stability, Zhu et al. [15] proposed the affine ICP algorithm with bidirectional distance to solve the affine registration problem. Meanwhile, Amberg et al. [16] presented to employ a locally affine regularization which assigned an affine transformation to each vertex and minimized the difference of neighboring vertices transformation. The above-mentioned methods about scaling and affine registration based on ICP have high precision and fast speed, but the above registration approaches greatly depending on the initial values are easily trapped into local minima and not robust enough.

As a consequence, the approaches based on the softassign method emerged. For instance, the thin plate spline-robust point matching (TPS-RPM) algorithm used TPS as the parameterization of the non-rigid spatial mapping and the softassign for the correspondence [17], but the searching speed was very slow. Meanwhile, in the non-rigid case, the coherent point drift (CPD) [18] imposed the coherence constraint by regularizing the displacement field and using the variational calculus to derive the optimal transformation. Moreover, some scholars extended to graph matching methods, and applied them to correspondence detection of general images [19], [20]. However, the above-mentioned methods are effective for the images that have similar structures or appearances, but when the images bear large deformations, the registration doesn’t accomplish satisfactory results.

Meanwhile, there are approaches to deal with the large deformation problem. Jia et al. [21] proposed the tree-based approach to handle this case, but it employed the mean squared difference of the intensity differences and was used primarily for image segmentation which didn’t meet our requirement of point set registration. Moreover, Ying et al. [22] proposed to use intensity differences to build the graph, and then apply a groupwise nonlinear registration method for image registration. Different from these methods, we propose the tree-based registration method based on the novel shape similarity to realize the optimal alignment of point sets with large differences. Moreover, the shape similarity is measured by the affine ICP algorithm with bidirectional distance which could avoid the ill-posed phenomenon that the affine transformation is close to zero. And then the shape similarity is used to build the tree with the nodes standing for model or subject images and the edges connecting model/subject image pairs with small shape differences. Point sets with small structural discrepancies are added to the tree heuristically until all subjects are constructed in the tree. By making full use of the information from image distribution to progressively conduct the non-rigid registration, the subjects with large differences are accurately registered by the model. We comprehensively evaluate the proposed framework associated with the state-of-the-art non-rigid registration methods including TPS and CPD on the part B of CE-Shape-1 [23]. Consequently, it achieves significant improvement compared with the traditional TPS and CPD individually, in terms of accuracy.

This paper is organized as follows. In Section 2, the typical non-rigid algorithms TPS and CPD are briefly introduced. Following that is Section 3, where the specific process of the proposed framework is analyzed. Next, the framework combined with TPS and CPD is verified advantageously to the point sets of large deformations via the experiments in Section 4. Finally, the conclusion is drawn in Section 5.

Section snippets

The traditional non-rigid registration methods

In practice, the most conventional non-rigid registration methods including the TPS and CPD algorithms adopt softassign method based on the full probability. Among them, the TPS algorithm uses deterministic annealing and alternate updates for soft assignment and parameters estimation [17]. Meanwhile, the CPD algorithm regularizes the displacement field between two point sets following the motion coherence theory [18]. Next, we briefly introduce the two methods.

The proposed framework

As the database contains a considerable number of individual subjects for the same object, there is no defying the fact that some subjects bear large structural discrepancies with respect to the model. Therefore, it is a big challenge to accomplish the non-rigid registration of point sets with large deformations. For the purpose of handling this case, the large deformation can be progressively segmented into several small discrepancies through the heuristic tree. In this tree, the images with

Experimental results

In this part, we evaluate the proposed framework on the part B of CE-Shape-1[23]. As the TPS [17] and CPD [18] algorithms are the traditional non-rigid registration methods, the proposed framework is combined with TPS and CPD, and then they are named Tree-TPS and Tree-CPD, respectively, which are compared with the traditional TPS and CPD to demonstrate the effectiveness and the accuracy. We randomly select one image as the model node, and other images as subjects. All the tests are done by

Conclusion

In this paper, we propose the new heuristic tree-based matching framework to deal with the problem of non-rigid registration of point sets with large deformation. For this purpose, we propose to use the affine ICP algorithm to learn the image distribution, and then build the heuristic tree and complete the non-rigid registration. The experiments verify the robustness and precision of the framework compared with the traditional non-rigid registration methods. The main contributions of the

Acknowledgements

This work was supported by 973 Program under Grant no. 2015CB351703, the National Natural Science Foundation of China under Grant no. 91320301, the Natural Science Basic Research Plan in Shaanxi Province of China under Grant nos. 2014JM8336 and 2012JQ8032, and the Program of Introducing Talents of Discipline to University under Grant no. B13043.

Shaoyi Du received B.S. degrees both in computational mathematics and in computer science, M.S. degree in applied mathematics and Ph.D. degree in pattern recognition and intelligence system from Xi’an Jiaotong University, China in 2002, 2005 and 2009, respectively. He worked as a postdoctoral fellow in Xi’an Jiaotong University from 2009 to 2011 and visited University of North Carolina at Chapel Hill from 2013 to 2014. He is currently an associate professor of the Institute of Artificial

References (31)

  • S. Ying et al.

    Hierarchical unbiased graph shrinkage (hugs): a novel groupwise registration for large data set

    NeuroImage

    (2014)
  • Y. Gao et al.

    Hyperspectral Image Classification Through Bilayer Graph Based Learning

    IEEE Trans. Image Process.

    (2014)
  • Y. Gao et al.

    Visual-textual Joint Relevance Learning for Tag-based Social Image Search

    IEEE Trans. Image Process.

    (2013)
  • Z. Zhang

    Iterative point matching for registration of free-form curves and surfaces

    Int. J. Comput. Vision

    (1994)
  • P.J. Besl, N.D. McKay, Method for Registration of 3-D Shapes, in: Robotics-DL Tentative, 1992, pp....
  • Cited by (10)

    • Anatomical landmark detection on 3D human shapes by hierarchically utilizing multiple shape features

      2017, Neurocomputing
      Citation Excerpt :

      The key ingredient of the method is that it builds a common template shape in a groupwise way by taking all the training shapes into account. Of note, The registration technique provides an effective way for aligning two/multiple point clouds, and various improvements have been made to enhance its accuracy [22] and robustness [23,24]. However, since a pure point cloud ignores the information how neighboring points are connected, these methods usually face challenges under large shape deformation such as articulation.

    • Movable type printing identification based on Tangut characters registration

      2017, Neurocomputing
      Citation Excerpt :

      Even the words printed by the same movable character model may be distorted during the process of printing and the character image acquisition. To solve the problem of the characters image registration of large deformation, we segment the large deformation into several small differences along the tree/graph [14–16]. In a tree/graph structure, the character images with similar shape are connected together, while the others with big differences are linked indirectly through several layers.

    • Affine scale space: an affine invariant image structure to promote the detection of correspondences from stereo images

      2017, Neurocomputing
      Citation Excerpt :

      A scale space can be defined as a collection of several pre-smoothed images by different sized Gaussian kernels [4]. In general, scale space theory is a framework for multi-scale signal representation to handle with image structures at different scales, by employing a one-parameterized family of smoothed images [16]. This framework provides a scale-invariant representation [17], which is necessary to deal with variations that may occur in image data [18].

    • Joint Template Matching by Candidate Set Voting

      2018, Proceedings 2018 Chinese Automation Congress, CAC 2018
    View all citing articles on Scopus

    Shaoyi Du received B.S. degrees both in computational mathematics and in computer science, M.S. degree in applied mathematics and Ph.D. degree in pattern recognition and intelligence system from Xi’an Jiaotong University, China in 2002, 2005 and 2009, respectively. He worked as a postdoctoral fellow in Xi’an Jiaotong University from 2009 to 2011 and visited University of North Carolina at Chapel Hill from 2013 to 2014. He is currently an associate professor of the Institute of Artificial Intelligence and Robotics in Xi’an Jiaotong University. His research interests include pattern recognition, machine learning and computer vision.

    Juan Liu received Bachelor degree in automation from Xi’an Jiaotong University, China in 2013. She is currently a graduate student in the institute of Artificial Intelligence and Robotics in Xi’an Jiaotong University. Her research interests include mobile robot and image registration.

    Chunjia Zhang received Bachelor degree from Nanjing University of Technology, China in 2008. He is currently a Ph.D. Candidate in the institute of Artificial Intelligence and Robotics in Xi’an Jiaotong University. His research interests include mobile robot and image registration.

    Meifeng Xu received B.S. degrees in Clinical Medicine from Tongji Medical College of Huazhong University of Science and Technology, China in 2002. She studied Mohs surgery technique, facial reconstruction, laser surgery and other cosmetic procedures as a visiting scholar in Warren Skin Care Center, NJ, USA from 2013 to 2014. She is currently an attending physician in the Department of Dermatology, The Second Affiliated Hospital of Xi’an Jiaotong University. Her research interests include dermatology surgery technique and medical image analysis.

    Jianru Xue received the B.S. degree from Xi’an University of Technology in 1994, and received the M.S. and Ph.D. degrees from Xi’an Jiaotong University, Xi’an, China, in 1999 and 2003, respectively. He had worked in FujiXerox, Tokyo, Japan, from 2002 to 2003, and visited University of California, Los Angeles, from 2008 to 2009. He is currently a professor of Institute of Artificial Intelligence and Robotics of Xi’an Jiaotong University. His research field includes computer vision, visual navigation, and video coding based on analysis.

    View full text