Skip to main content
SpringerLink
Log in

Multi-part shape matching by simultaneous partial functional correspondence

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Non-rigid multi-part shape matching has proven to be essential and challenging in many applications. This paper analyzes the aforementioned problem and proposes a novel multi-part shape matching method to simultaneously compute correspondences between a full shape and its multiple parts undergoing a non-rigid deformation. The main idea is to simultaneously integrate the Hamiltonian eigenvalue equivalence strategy as a part regularization term being fully spectral with the partial functional map. Moreover, we introduce a new upsampling refinement approach based upon ZoomOut in conjunction with the regularized point-wise map recovery algorithm to obtain high-quality partial matches. Our method naturally handles various challenges and noise that commonly occur in real scans, like non-rigid deformations, strong partiality, topological noise, and symmetric ambiguity. Finally, we demonstrate superior qualitative and quantitative results on several datasets. We show that our method produces more accurate, smoother results than other competing methods in realistic scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Van, K.O., Zhang, H., Hamarneh, G., Cohen-Or, D.: A survey on shape correspondence. Comput. Gr. Forum 30(6), 1681–1707 (2011)

    Article  Google Scholar 

  2. Aigerman, N., Poranne, R., Lipman, Y.: Seamless surface mappings. ACM Trans. Gr. 34(4), 1–13 (2015)

    Article  MATH  Google Scholar 

  3. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., Seidel, H.P.: A statistical model of human pose and body shape. Comput. Gr. Forum 28(2), 337–346 (2009)

    Article  Google Scholar 

  4. Bogo, F., Romero, J., Loper, M., Black, MJ.: FAUST: dataset and evaluation for 3D mesh registration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3794–3801 (2014)

  5. Gasparetto, A., Torsello, A.: A statistical model of Riemannian metric variation for deformable shape analysis. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1219–1228 (2015)

  6. Johnson-Roberson, M., Bryson, M., Friedman, A., Pizarro, O., Troni, G., Ozog, P., Henderson, J.C.: High-resolution underwater robotic vision-based mapping and three-dimensional reconstruction for archaeology. J. Field Robot. 34(4), 625–643 (2017)

    Article  Google Scholar 

  7. Papaioannou, G., Karabassi, E.A., Theoharis, T.: Virtual archaeologist: assembling the past. IEEE Comput. Gr. Appl. 21(2), 53–59 (2001)

    Article  Google Scholar 

  8. Huang, Q. X., Flöry, S., Gelfand, N., Hofer, M., Pottmann, H.: Reassembling fractured objects by geometric matching. In: Proceedings of the ACM Siggraph, pp. 569–578 (2006)

  9. Nguyen, A., Ben-Chen, M., Welnicka, K., Ye, Y., Guibas, L.: An optimization approach to improving collections of shape maps. Comput. Gr. Forum 30(5), 1481–1491 (2011)

    Article  Google Scholar 

  10. Cosmo, L., Rodolà, E., Albarelli, A., Mémoli, F., Cremers, D.: Consistent partial matching of shape collections via sparse modeling. Comput. Gr. Forum 36(1), 209–221 (2017)

    Article  Google Scholar 

  11. Li, H., Sumner, R.W., Pauly, M.: Global correspondence optimization for non-rigid registration of depth scans. Comput. Gr. Forum 27(5), 1421–1430 (2008)

    Article  Google Scholar 

  12. Litany, O., Rodolà, E., Bronstein, A.M., Bronstein, M.M.: Fully spectral partial shape matching. Comput. Gr. Forum 36(2), 247–258 (2017)

    Article  Google Scholar 

  13. Rodolà, E., Möller, M., Cremers, D.: Regularized point-wise map recovery from functional correspondence. Comput. Gr. Forum 36(8), 700–711 (2017)

    Article  Google Scholar 

  14. Melzi, S., Ren, J., Rodolà, E., Sharma, A., Wonka, P., Ovsjanikov, M.: Zoomout: spectral upsampling for efficient shape correspondence. ACM Trans. Graph. 38(6), 155–170 (2019)

    Article  Google Scholar 

  15. Biasotti, S., Cerri, A., Bronstein, A.M., Bronstein, M.M.: Recent trends, applications, and perspectives in 3D shape similarity assessment. Comput. Gr. Forum 35(6), 87–119 (2016)

    Article  Google Scholar 

  16. Sahillioğlu, Y.: Recent advances in shape correspondence. Vis. Comput. 36(8), 1705–1721 (2020)

    Article  Google Scholar 

  17. Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. ACM Trans. Graph. 31(4), 30–41 (2012)

    Article  Google Scholar 

  18. Pokrass, J., Bronstein, A.M., Bronstein, M.M., Sprechmann, P., Sapiro, G.: Sparse modeling of intrinsic correspondences. Comput. Gr. Forum 32(2pt4), 459–468 (2013)

    Article  MATH  Google Scholar 

  19. Gasparetto, A., Cosmo, L., Rodolà, E., Bronstein, M. M., Torsello, A.: Spatial maps: From low rank spectral to sparse spatial functional representations. In: Proceedings of the international conference on 3D vision, pp. 477–485 (2017)

  20. Aflalo, Y., Dubrovina, A., Kimmel, R.: Spectral generalized multi-dimensional scaling. Int. J. Comput. Vis. 118(3), 380–392 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  21. Yang, J., Yan, H.: An algorithm for calculating shape correspondences using functional maps by calibrating base matrix of 3D shapes. Geomat. Inf. Sci. Wuhan Univ. 43(10), 1518–1525 (2018)

    Google Scholar 

  22. Eynard, D., Rodolà, E., Glashoff, K., Bronstein, M. M.: Coupled functional maps. In: Proceedings of the international conference on 3D vision, pp. 399–407 (2016)

  23. Ezuz, D., Solomon, J., Ben-Chen, M.: Reversible harmonic maps between discrete surfaces. ACM Trans. Gr. 38(2), 1–12 (2019)

    Article  Google Scholar 

  24. Ezuz, D., Heeren, B., Azencot, O., Rumpf, M., Ben-Chen, M.: Elastic correspondence between triangle meshes. Comput. Graph. Forum 38(2), 121–134 (2019)

    Article  Google Scholar 

  25. Melzi, S., Rodolà, E., Castellani, U., Bronstein, M.M.: Localized manifold harmonics for spectral shape analysis. Comput. Gr. Forum 37(6), 20–34 (2018)

    Article  Google Scholar 

  26. Wang, Y., Ben-Chen, M., Polterovich, I., Solomon, J.: Steklov spectral geometry for extrinsic shape analysis. ACM Trans. Gr. 38(1), 1–21 (2018)

    Article  Google Scholar 

  27. Nogneng, D., Ovsjanikov, M.: Informative descriptor preservation via commutativity for shape matching. Comput. Gr. Forum 36(2), 259–267 (2017)

    Article  Google Scholar 

  28. Nogneng, D., Melzi, S., Rodolà, E., Castellani, U., Bronstein, M.M., Ovsjanikov, M.: Improved functional mappings via product preservation. Comput. Gr. Forum 37(2), 179–190 (2018)

    Article  Google Scholar 

  29. Aiger, D., Mitra, N. J., Cohen-Or, D.: 4-points congruent sets for robust pairwise surface registration. In: Proceedings of the ACM Siggraph, pp. 1–10 (2008)

  30. Albarelli, A., Rodolà, E., Torsello, A.: Fast and accurate surface alignment through an isometry-enforcing game. Pattern Recognit. 48(7), 2209–2226 (2015)

    Article  MATH  Google Scholar 

  31. Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Generalized multi-dimensional scaling: a framework for isometry-invariant partial surface matching. Proc. Natl. Acad. Sci. 103(5), 1168–1172 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  32. Bronstein, A.M., Bronstein, M.M., Bruckstein, A.M., Kimmel, R.: Partial similarity of objects, or how to compare a centaur to a horse. Int. J. Comput. Vis. 84(2), 163–183 (2009)

    Article  Google Scholar 

  33. Windheuser, T., Schlickwei, U., Schimdt, F.R., Cremers, D.: Large-scale integer linear programming for orientation preserving 3d shape matching. Comput. Graph. Forum 30(5), 1471–1480 (2011)

    Article  Google Scholar 

  34. Sahillioğlu, Y., Yemez, Y.: Partial 3-D correspondence from shape extremities. Comput. Graph. Forum 33(6), 63–76 (2014)

    Article  Google Scholar 

  35. Masci, J., Boscaini, D., Bronstein, M. M., Vandergheynst, P.: Geodesic convolutional neural networks on Riemannian manifolds. In: Proceedings of the IEEE international conference on computer vision workshops, pp. 37–45 (2015)

  36. Wei, L., Huang, Q., Ceylan, D., Vouga, E., Li, H.: Dense human body correspondences using convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1544–1553 (2016)

  37. Kovnatsky, A., Bronstein, M. M, Bresson, X., Vandergheynst, P.: Functional correspondence by matrix completion. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 905–914 (2015)

  38. Rodolà, E., Cosmo, L., Bronstein, M.M., Torsello, A., Cremers, D.: Partial functional correspondence. Comput. Gr. Forum 36(1), 222–236 (2017)

    Article  Google Scholar 

  39. Wu, Y., Yang, J., Zhao, J.: Partial 3D shape functional correspondence via fully spectral eigenvalue alignment and upsampling refinement. Comput. Gr. 92, 99–113 (2020)

    Article  Google Scholar 

  40. Torsello, A., Rodolà, E., Albarelli, A.: Multiview registration via graph diffusion of dual quaternions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2441–2448 (2011)

  41. Litany, O., Bronstein, A. M., Bronstein, M. M.: Putting the pieces together: Regularized multi-part shape matching. In: Proceedings of the European conference on computer vision, pp. 1–11 (2012)

  42. Litany, O., Rodolà, E., Bronstein, A.M., Bronstein, M.M., Cremers, D.: Non-rigid puzzles. Comput. Gr. Forum 35(5), 135–143 (2016)

    Article  Google Scholar 

  43. Cosmo, L., Rodolà, E., Masci, J., Torsello, A., Bronstein, M. M.: Matching deformable objects in clutter. In: Proceedings of the international conference on 3D vision, pp. 1–10 (2016)

  44. Sahillioğlu, Y., Yemez, Y.: Multiple shape correspondence by dynamic programming. Comput. Gr. Forum 33(7), 121–130 (2014)

    Article  Google Scholar 

  45. Huang, Q., Wang, F., Guibas, L.: Functional map networks for analyzing and exploring large shape collections. ACM Trans. Gr. 33(4), 1–11 (2014)

    MATH  Google Scholar 

  46. Yang, J., Lei, M.: Correspondence calculation of model cluster by functional mapping combined with cycle-consistency constraints. Laser Optoelectron. Progress. 056(008), 115–125 (2019)

    Google Scholar 

  47. Cohen, A., Ben-Chen, M.: Robust shape collection matching and correspondence from shape differences. Comput. Gr. Forum 39(2), 555–568 (2020)

    Article  Google Scholar 

  48. Huang, R., Ren, J., Wonka, P., Ovsjanikov, M.: Consistent zoomout: efficient spectral map synchronization. Comput. Gr. Forum 39(5), 265–278 (2020)

    Article  Google Scholar 

  49. Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. Comput. Gr. Forum 28(5), 1383–1392 (2009)

    Article  Google Scholar 

  50. Aubry, M., Schlickewei, U., Cremers, D.: The wave kernel signature: a quantum mechanical approach to shape analysis. In: Proceedings of the IEEE international conference on computer vision workshops, pp. 1626–1633 (2011)

  51. Cosmo, L., Minello, G., Bronstein, M. M., Rossi, L., Torsello, A.: The average mixing kernel signature. In: Proceedings of the European conference on computer vision, pp. 1–17 (2020)

  52. Halimi, O., Litany, O., Rodolà, E., Bronstein, A. M., Kimmel, R.: Unsupervised learning of dense shape correspondence. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4370–4379 (2019)

  53. Roufosse, J. M., Sharma, A., Ovsjanikov, M.: Unsupervised deep learning for structured shape matching. In: Proceedings of the IEEE international conference on computer vision, pp. 1617–1627 (2019)

  54. Ren, J., Panine, M., Wonka, P., Ovsjanikov, M.: Structured regularization of functional map computations. Comput. Gr. Forum 38(5), 39–53 (2019)

    Article  Google Scholar 

  55. Ezuz, D., Ben-Chen, M.: Deblurring and denoising of maps between shapes. Comput. Gr. Forum 36(5), 165–174 (2017)

    Article  Google Scholar 

  56. Kovnatsky, A., Bronstein, M.M., Bronstein, A.M., Glashoff, K., Kimmel, R.: Coupled quasi-harmonic bases. Comput. Gr. Forum 32(2pt4), 439–448 (2013)

    Article  Google Scholar 

  57. Rampini, A., Tallini, I., Ovsjanikov, M., Bronstein, A. M., Rodolà, E.: Correspondence-free region localization for partial shape similarity via Hamiltonian spectrum alignment. In: Proceedings of the international conference on 3D vision, pp. 37–46 (2019)

  58. Choukroun, Y., Shtern, A., Bronstein, A. M., Kimmel, R.: Hamiltonian operator for spectral shape analysis. In: Proceedings of the IEEE transactions on visualization and computer graphics, pp. 1–12 (2018)

  59. Postolache, E., Fumero, M., Cosmo, L., Rodolà, E.: A parametric analysis of discrete Hamiltonian functional maps. Comput. Gr. Forum 39(5), 103–118 (2020)

    Article  Google Scholar 

  60. Rustamov, R. M.: Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In: Proceedings of the Eurographics Association, pp. 225–233 (2007)

  61. Bronstein, A. M., Bronstein, M. M.: Not only size matters: regularized partial matching of nonrigid shapes. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition workshops, pp. 1–6 (2008)

  62. Kovnatsky, A., Glashoff, K., Bronstein, M. M.: MADMM: a generic algorithm for non-smooth optimization on manifolds. In: Proceedings of the European conference on computer vision, pp. 680–696 (2016)

  63. Rodolà, E., Moeller, M., Cremers, D.: Point-wise map recovery and refinement from functional correspondence. In: Proceedings of the international symposium on vision, modeling, and visualization, pp. 25–32 (2015)

  64. Yuille, A. L., Grzywacz, N. M.: The motion coherence theory. In: Proceedings of the IEEE international conference on computer vision, pp. 26–33 (1988)

  65. Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2262–2275 (2010)

    Article  Google Scholar 

  66. Tombari, F., Salti, S., Di, S. L.: Unique signatures of histograms for local surface description. In: Proceedings of the European conference on computer vision, pp. 356–369 (2010)

  67. Kim, V.G., Lipman, Y., Funkhouser, T.: Blended intrinsic maps. ACM Trans. Gr. 30(4), 1–12 (2011)

    Article  Google Scholar 

  68. Cosmo, L., Rodolà, E., Bronstein, M. M., Torsello, A., Cremers, D., Sahillioğlu, Y.: Shrec16: partial matching of deformable shapes. In: Proceedings of the Eurographics workshop on 3D object retrieval, pp. 61–67 (2016)

  69. Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Numerical Geometry of Non-Rigid Shapes. Springer Science & Business Media, Berlin (2008)

    MATH  Google Scholar 

  70. Läner, Z., Rodolà, E., Bronstein, M. M., Cremers, D., Burghard, O., Cosmo, L., Sahillioğlu, Y.: Shrec16: matching of deformable shapes with topological noise. In: Proceedings of the Eurographics workshop on 3D object retrieval, pp. 55–60 (2016)

  71. Rodolà, E., Rota, BuloS., Windheuser, T., Vestner, M., Cremers, D.: Dense non-rigid shape correspondence using random forests. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4177–4184 (2014)

  72. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: SCAPE: shape completion and animation of people. ACM Trans. Gr. 24(3), 408–416 (2005)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China [Grant Number 61862039].

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou, China

    Yan Wu & Jun Yang

  2. School of Electronic and Information Engineering, Fujian Polytechnic Normal University, Fuqing, China

    Yan Wu

Authors
  1. Yan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Yang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Y., Yang, J. Multi-part shape matching by simultaneous partial functional correspondence. Vis Comput 39, 393–412 (2023). https://doi.org/10.1007/s00371-021-02337-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02337-6

Keywords

Search

Navigation