Skip to main content
SpringerLink
Log in

Topology-aware non-rigid point set registration via global–local topology preservation

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

We propose a new topology-aware point set registration algorithm which can cope with multi-part articulated and non-rigid deformations. Point set registration is formulated as a maximum likelihood (ML) estimation problem where two topologically complementary constraints are jointly optimized in a probabilistic framework. The first is coherent point drift that keeps the overall spatial connectivity and associativity by moving the point set collectively and coherently. The second is local linear embedding that preserves the local topological structure during registration. Hence, the new algorithm is called global–local topology preservation (GLTP). Without any pre-segmentation and correspondence initialization, GLTP is particularly useful and effective in dealing with complex shape matching with non-coherent and non-rigid local deformations at different parts of a point set. We have derived the expectation maximization algorithm for the ML optimization constrained with both regularization terms. Experimental results on a large set of 2D and 3D examples show the advantages and robustness of GLTP over existing algorithms in the presence of outliers, noise and missing data, especially in the case of articulated non-rigid transformations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. https://github.com/songdevelop/gltp.

References

  1. Ye, M., Wang, X., Yang, R., Liu, R., Pollefeys, M.: Accurate 3D pose estimation from a single depth image. In: Proceedings of IEEE International Conference on Computer Vision, pp. 731–738 (2011)

  2. Weiss, A., Hirshberg, D., Black, M.J.: Home 3D body scans from noisy image and range data. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1951–1958 (2011)

  3. Park, S.-Y., Baek, J., Moon, J.: Hand-held 3D scanning based on coarse and fine registration of multiple range images. Mach. Vis. Appl. 22(3), 563–579 (2011)

    Google Scholar 

  4. Park, S.-Y., Choi, S.-I., Kim, J., Chae, J.S.: Real-time 3D registration using GPU. Mach. Vis. Appl. 22(5), 837–850 (2011)

    Article  Google Scholar 

  5. Pribanić, T., Diez, Y., Roure, F., Salvi, J.: An efficient surface registration using smartphone. Mach. Vis. Appl. 27(4), 559–576 (2016)

    Article  Google Scholar 

  6. Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)

    Article  Google Scholar 

  7. Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. Int. J. Comput. Vis. 13(2), 119–152 (1994)

    Article  Google Scholar 

  8. Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings of International Conference on 3D Digital Imaging and Modeling (3DIM) (2001)

  9. Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24, 325–376 (1992)

    Article  Google Scholar 

  10. Makadia, A., Patterson, A., Daniilidis, K.: Fully automatic registration of 3S point clouds. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1297–1304 (2006)

  11. Chui, H., Rangarajan, A.: A feature registration framework using mixture models. In: Proceedings of IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, pp. 190–197 (2000)

  12. Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. Comput. Vis. Image Underst. 89(2–3), 114–141 (2003)

    Article  MATH  Google Scholar 

  13. Myronenko, A., Song, X., Carreira-Perpinan, M.A.: Non-rigid point set registration: coherent point drift (CPD). In: Proceedings of Advances in Neural Information Processing Systems, pp. 1009–1016 (2006)

  14. Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2262–2275 (2010)

    Article  Google Scholar 

  15. Jian, B., Vemuri, B.C.: A robust algorithm for point set registration using mixture of Gaussians. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1246–1251 (2005)

  16. Breitenreicher, D., Schnörr, C.: Robust 3D object registration without explicit correspondence using geometric integration. Mach. Vis. Appl. 21(5), 601–611 (2010)

    Article  Google Scholar 

  17. Jian, B., Vemuri, B.C.: Robust point set registration using Gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1633–45 (2011)

    Article  Google Scholar 

  18. Gerogiannis, D., Nikou, C., Likas, A.: The mixtures of Students’ t-distributions as a robust framework for rigid registration. Image Vis. Comput. 27(9), 1285–1294 (2009)

    Article  Google Scholar 

  19. Sfikas, G., Nikou, C., Galatsanos, N.P.: Edge preserving spatially varying mixtures for image segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2008)

  20. Bishop, C.M., Svensén, M.: Robust Bayesian mixture modelling. Neurocomputing 64, 235–252 (2005)

    Article  Google Scholar 

  21. Zhou, Z., Zheng, J., Dai, Y., Zhou, Z., Chen, S.: Robust non-rigid point set registration using Student’s-t mixture model. PLoS ONE 9, e91381 (2014)

    Article  Google Scholar 

  22. Tsin, Y., Kanade, T.: A correlation-based approach to robust point set registration. In: Proceedings of European Conference on Computer Vision, pp. 558–569 (2004)

  23. Ding, M., Fan, G.: Articulated and generalized Gaussian kernel correlation for human pose estimation. IEEE Trans. Image Process. 25(2), 776–789 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Wang, G., Wang, Z., Chen, Y., Zhao, W.: A robust non-rigid point set registration method based on asymmetric Gaussian representation. Comput. Vis. Image Underst. 141, 67–80 (2015)

    Article  Google Scholar 

  25. Kato, T., Omachi, S., Aso, H.: Asymmetric Gaussian and its application to pattern recognition. In: Structural, Syntactic, and Statistical Pattern Recognition, pp. 227–242 (2002)

  26. Wang, G., Zhou, Q., Chen, Y.: Robust non-rigid point set registration using spatially constrained Gaussian fields. IEEE Trans. Image Process. 26(4), 1759–1769 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  27. Boughorbel, F., Mercimek, M., Koschan, A., Abidi, M.: A new method for the registration of three-dimensional point-sets: the Gaussian fields framework. Image Vis. Comput. 28(1), 124–137 (2010)

    Article  Google Scholar 

  28. Ma, J., Qiu, W., Zhao, J., Ma, Y., Yuille, A.L., Tu, Z.: Robust L2E estimation of transformation for non-rigid registration. IEEE Trans. Signal Process. 63(5), 1115–1129 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  29. Ma, J., Zhao, J., Yuille, A.L.: Non-rigid point set registration by preserving global and local structures. IEEE Trans. Image Process. 25(1), 53–64 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  30. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  31. Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)

    Article  Google Scholar 

  32. Zheng, Y., Doermann, D.: Robust point matching for nonrigid shapes by preserving local neighborhood structures. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 643–649 (2006)

    Article  Google Scholar 

  33. Ma, J., Zhao, J., Tian, J., Tu, Z., Yuille, A.L.: Robust estimation of nonrigid transformation for point set registration. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2147–2154 (2013)

  34. Ma, J., Zhao, J., Tian, J.W., Yuille, A.L., Tu, Z.W.: Robust point matching via vector field consensus. IEEE Trans. Image Process. 23(4), 1706–1721 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  35. Ge, S., Fan, G.: Non-rigid articulated point set registration with local structure preservation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2015)

  36. Panaganti, V., Aravind, R.: Robust nonrigid point set registration using Graph-Laplacian regularization. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision, pp. 1137–1144 (2015)

  37. Pellegrini, S., Schindler, K., Nardi, D.: A generalization of the ICP algorithm for articulated bodies. In: Proceedings of British Machine Vision Conference, pp. 87.1–87.10 (2008)

  38. Horaud, R., Forbes, F., Yguel, M., Dewaele, G., Zhang, J.: Rigid and articulated point registration with expectation conditional maximization. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 587–602 (2011)

    Article  Google Scholar 

  39. Ye, M., Yang, R.: Real-time simultaneous pose and shape estimation for articulated objects using a single depth camera. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2353–2360 (2014)

  40. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)

    Article  Google Scholar 

  41. Chappelow, J., Madabhushi, A., Rosen, M., Tomaszeweski, J., Feldman, M.: Multimodal image registration of ex vivo 4 Tesla MRI with whole mount histology for prostate cancer detection. In: Medical Imaging 2007: Image Processing, Proceeddings of SPIE 6512, 65121S (2007)

  42. Aljabar, P., Robin W., Daniel, R.: Manifold learning for medical image registration, segmentation, and classification. In: Machine Learning in Computer-Aided Diagnosis: Medical Imaging Intelligence and Analysis, pp. 351–372. IGI Global (2012)

  43. Mateus, D., Cuzzolin, F., Horaud, R., Boyer, E.: Articulated shape matching using locally linear embedding and orthogonal alignment. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, pp. 1–8. IEEE (2007)

  44. Ge, S., Fan, G., Ding, M.: Non-rigid point set registration with global–local topology preservation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 245–251 (2014)

  45. Ge, S., Fan, G.: Articulated non-rigid point set registration for human pose estimation from 3D sensors. Sensors 15(7), 15218 (2015)

    Article  Google Scholar 

  46. Ge, S., Fan, G.: Sequential non-rigid point registration for 3D human pose tracking. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 1105–1109. IEEE (2015)

  47. Ge, S. Fan, G.: Non-rigid articulated point set registration for human pose estimation. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 94–101. IEEE (2015)

  48. de Sousa, S., Kropatsch, W.G.: Graph-based point drift: graph centrality on the registration of point-sets. Pattern Recognit. 48(2), 368–379 (2015)

    Article  MATH  Google Scholar 

  49. Li, X., Yankeelov, T.E., Peterson, T.E., Gore, J.C., Dawant, B.M.: Constrained non-rigid registration for whole body image registration: method and validation. In: Proceedings of SPIE, Medical Imaging: Image Processing, pp. 651202.1–651202.8 (2007)

  50. Mateus, D., Horaud, R., Knossow, D., Cuzzolin, F., Boyer, E.: Articulated shape matching using Laplacian eigenfunctions and unsupervised point registration. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

  51. Ye, M., Shen, Y., Du, C., Pan, Z., Yang, R.: Real-time simultaneous pose and shape estimation for articulated objects with a single depth camera. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1517–1532 (2016)

    Article  Google Scholar 

  52. Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, New York (1995)

    MATH  Google Scholar 

  53. Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Comput. 7, 219–269 (1995)

    Article  Google Scholar 

  54. Micchelli, C.A., Pontil, M.A.: On learning vector-valued functions. Neural Comput. 17(1), 177–204 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  55. Zhao, J., Ma, J., Tian, J., Ma, J., Zhang, D.: A robust method for vector field learning with application to mismatch removing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2977–2984 (2011)

  56. Yuille, A.L., Grzywacz, N.M.: A mathematical analysis of the motion coherence theory. Int. J. Comput. Vis. 3(2), 155–175 (1989)

    Article  Google Scholar 

  57. Ge, S., Fan, G.: Articulated non-rigid point set registration for human pose estimation from 3D sensors. Sensors 15(7), 15218–15245 (2015)

    Article  Google Scholar 

  58. Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. ACM Trans. Graph. 23(3), 399–405 (2004)

    Article  Google Scholar 

  59. Chen, X., Golovinskiy, A., Funkhouser, T.: A benchmark for 3D mesh segmentation. In: ACM Transactions on Graphics (Proceedings of the SIGGRAPH), vol. 28, no. 3 (2009)

  60. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: SCAPE: shape completion and animation of people. ACM Trans. Graph. 24, 408–416 (2005)

    Article  Google Scholar 

  61. Christensen, G.E., Johnson, H.J.: Invertibility and transitivity analysis for nonrigid image registration. J. Electron. Imaging 12(1), 106–117 (2003)

    Article  Google Scholar 

  62. Datteri, R.D., Liu, Y., D’Haese, P.-F., Dawant, B.M.: Validation of a nonrigid registration error detection algorithm using clinical MRI brain data. IEEE Trans. Med. Imaging 34(1), 86–96 (2015)

    Article  Google Scholar 

  63. Greengard, L., Strain, J.: The fast Gauss transform. SIAM J. Sci. Stat. Comput. 12(1), 79–94 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  64. Jia, Z., Chang, Y.-J., Lin, T.-H., Chen, T.: Dense interpolation of 3D points based on surface and color. In: Proceedings of IEEE International Conference on Image Processing, pp. 869–872 (2011)

Download references

Acknowledgements

This work is supported in part by the Oklahoma Center for the Advancement of Science and Technology (OCAST) under Grants HR12-030 and HR18-069 and the National Science Foundation (NSF) under Grant NRI-1427345.

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, Oklahoma State University, Stillwater, OK, 74078, USA

    Song Ge & Guoliang Fan

  2. School of Automation and Information Engineering, Xi’an University of Technology, Xi’an, 710048, China

    Guoliang Fan

Authors
  1. Song Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guoliang Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guoliang Fan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ge, S., Fan, G. Topology-aware non-rigid point set registration via global–local topology preservation. Machine Vision and Applications 30, 717–735 (2019). https://doi.org/10.1007/s00138-019-01024-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-019-01024-w

Keywords

Search

Navigation