Skip to main content
SpringerLink
Log in

Generalized unsupervised functional map learning for dense correspondence

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Inspired by deep functional map methods, we present a generalized unsupervised functional map learning approach for arbitrary 3D shape correspondence. Unlike prior methods, they either require extensive data training or rely on input features; our model directly operates on point clouds and learns both deep features and optimized basis function without the constraint of geometric connectivity and the assumption of isometry. We propose a novel scheme that combines structural embedding based on Mahalanobis distance and locally linear embedding to learn the optimized feature basis. Furthermore, the constructed shape descriptors effectively optimize the estimation of functional map and dense correspondence through a tri-level regularization mechanism that enforces penalties on global structural properties, representation error and pair-wise Mahalanobis distance distortion, which significantly improves the performance of unsupervised learning. Extensive experiments in shape matching show that our method can learn from less training data and has better generalization ability compared with the state-of-the-art supervised and unsupervised methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. Comput. Graph. Forum. 28, 1383–1392 (2009)

    Article  Google Scholar 

  2. Aubry, M., Schlickewei, U., Cremers, D.: The wave kernel signature: a quantum mechanical approach to shape analysis. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1626–1633. IEEE (2011)

  3. Bronstein, M.M., Kokkinos, I.: Scale-invariant heat kernel signatures for non-rigid shape recognition. In: Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1704–1711. IEEE (2010)

  4. Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Proceedings of the European Conference on Computer Vision, pp. 356–369. Springer (2010)

  5. Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. In: TOG (2012)

  6. Litany,O., Remez, T., Rodolà, E., Bronstein, A., Bronstein, M.: Deep functional maps: structured prediction for dense shape correspondence. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 5659–5667 (2017)

  7. Salti, S., Tombari, F., Di Stefano, L.: Shot: unique signatures of histograms for surface and texture description. Comput. Vis. Image Underst. 125, 251–264 (2014)

    Article  Google Scholar 

  8. Halimi, O., Litany, O., Rodola, E., Bronstein, A.M., Kimmel, R.: Unsupervised learning of dense shape correspondence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4370–4379 (2019)

  9. Roufosse, J.-M., Sharma, A., Ovsjanikov, M.: Unsupervised deep learning for structured shape matching. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1617–1627 (2019)

  10. Aygün, M., Lhner, Z., Cremers, D.: Unsupervised Dense Shape Correspondence using Heat Kernels. 2020.

  11. Kim, V.G., Li, W., Mitra, N.J., Di Verdi, S., Funkhouser, T.: Exploring collections of 3D models using fuzzy correspondences. ACM Trans. Graph (TOG) 31(4), 54:1-54:11 (2012)

    Article  Google Scholar 

  12. Küpü, E., Yemez, Y.: Diffusion-based isometric depth correspondence. Comput. Vis. Image Underst. 189, 102808 (2019)

    Article  Google Scholar 

  13. Rustamov, R.M.: Laplace-beltrami eigenfunctions for deformation invariant shape representation. In: Proceedings of the Fifth Eurographics Symposium on Geometry Processing, pp. 225–233. Eurographics Association, (2007)

  14. Litman, R., Bronstein, A.M.: Learning spectral descriptors for deformable shape correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 171–180 (2013)

    Article  Google Scholar 

  15. Boscaini, D., Masci, J., Rodolà, E., Bronstein, M.: Learning shape correspondence with anisotropic convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 3189–3197 (2016)

  16. Qi, C.R., Su, H., Mo, K., et al.: PointNet: deep learning on point sets for 3D classification and segmentation. In: IEEE (2017).

  17. Qi, C.R., Yi, L., Su, H., et al.: PointNet++: Deep hierarchical feature learning on point sets in a metric space. (2017)

  18. Wu, W., Qi, Z., Li, F.: PointConv: deep convolutional networks on 3D point clouds. In: CVPR (2019)

  19. Thomas, H., Qi, C. R., Deschaud, J. et al.: KPConv: flexible and deformable convolution for point clouds. In: Proceedings of the ICCV (2019)

  20. Huang, Q., Wang, F., Guibas, L.: Functional map networks for analyzing and exploring large shape collections. In: TOG (2014)

  21. Boscaini, D., Masci, J., Rodolà, E., Bronstein, M.M., Cremers, D.: Anisotropic diffusion descriptors. In: Computer Graphics Forum, vol. 35, pp. 431–441. Wiley Online Library (2016)

  22. Burghard, O., Dieckmann, A., Klein, R.: Embedding shapes with Green’s functions for global shape matching. Comput. Graph. 68, 1–10 (2017)

    Article  Google Scholar 

  23. Ren, J., Poulenard, A., Wonka, P., Ovsjanikov, M.: Continuous and orientation-preserving correspon dences via functional maps. ACM Trans. Graph. (TOG) 37, 6 (2018)

    Article  Google Scholar 

  24. Rodola, E., Cosmo, L., Bronstein, M.M., Torsello, A., Cremers, D.: Partial functional correspondence. In: Computer Graphics Forum, vol. 36, pp. 222–236. Wiley Online Library (2017)

  25. Ren, J., Panine, M., Wonka, P., et al. Structured Regularization of Functional Map Computations (2020)

  26. Kovnatsky, A., Bronstein, M.M., Bronstein, A.M., Glashoff, K., Kimmel, R.: Coupled quasi-harmonic bases. Comput. Graph. Forum 32, 439–448 (2013)

    Article  Google Scholar 

  27. Kovnatsky, A., Bronstein, M.M., Bresson, X., et al.: Functional correspondence by matrix completion. IEEE (2015)

  28. Rodolà, E., Moeller, M., Cremers, D.: Point-wise map recovery and refinement from functional correspondence. Comput. Sci. (2015)

  29. Ezuz, D., Ben-Chen, M.: Deblurring and denoising of maps between shapes. Comput. Graph. Forum 36(5), 165–174 (2017)

    Article  Google Scholar 

  30. Aflalo, Y., Brezis, H., Kimmel R.: On the optimality of shape and data representation in the spectral domain. Siam J. Imaging Sci. 8(2) (2014)

  31. Tenenbaum, J., Silva, V., Langford, C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)

    Article  Google Scholar 

  32. Muruganathan, S., N., Devarajan, D., and Chitra, et al.: Shape retrieval through mahalanobis distance with shortest augmenting path algorithm. J. Comput. Sci. 10(4), 552–562 (2014)

    Article  Google Scholar 

  33. Roweis, S.T.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)

    Article  Google Scholar 

  34. Goyal, P., Ferrara, E.: Graph embedding techniques, applications, and performance: a survey. Knowl Based Syst (2017)

  35. Abadi, M., Agarwal, A., Barham, P., et al.: Tensor Flow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (2015)

  36. Aflflalo, Y., Dubrovina, A., Kimmel, R.: Spectral generalized multi-dimensional scaling. IJCV 118(3), 380–392 (2016)

    Article  MathSciNet  Google Scholar 

  37. Donati, N., Sharma, A., Ovsjanikov, M.: Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence. In: IEEE (2020)

  38. Groueix, T., Fisher, M., Kim, V. et al.: 3D-CODED : 3D Correspondences by Deep Deformation (2018)

  39. Masci, J., Boscaini, D., Bronstein, M.M. et al.: Geodesic convolutional neural networks on Riemannian manifolds. In: Proceedings of the IEEE International Conference on Computer Vision Workshop (ICCVW). IEEE (2016)

  40. Zeng, Y., Qian, Y., Zhu, Z., Hou, J., Yuan, H., He, Y.:CorrNet3D: unsupervised end-to-end learning of dense correspondence for 3D point clouds. In: CVPR, pp. 6048–6057 (2021)

  41. Vestner, M., Litman, R., Rodolà, E. et al.: Product Manifold Filter: Non-Rigid shape correspondence via kernel density estimation in the product space. IEEE (2017)

  42. Ye, J., Yu, Y.: A fast modal space transform for robust nonrigid shape retrieval. Visual Comput. 32(5), 553–568 (2015)

    Article  Google Scholar 

  43. Yu, F., Liu, K., Zhang, Y., Zhu, C., Xu, K.: PartNet: a large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In: Proceedings of the CVPR (2019)

  44. Yi, L., Zhao, W., Wang, H., Sung, M., Guibas, L.: StructureNet: Hierarchical graph networks for 3D shape generation. In: Proceedings of the Siggraph Asia (2019)

Download references

Acknowledgements

We would like to thank the anonymous reviewers for their helpful comments. The research presented in this paper is supported by a grant from NSFC (61702246), grants from research projects of Liaoning province (2019lsktyb-084, LJ2020015, 2020JH4/10100045) and a fund of Dalian Science and Technology (2019J12GX038).

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Liaoning Normal University, Dalian, 116021, China

    Li Han, Xue Shi, Jinhai He, Huiwen Ma & Feng Dou

  2. Department of Mathematics, University of California, Irvine, CA, USA

    Hongkai Zhao

Authors
  1. Li Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xue Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinhai He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huiwen Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Feng Dou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hongkai Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Han.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, L., Shi, X., He, J. et al. Generalized unsupervised functional map learning for dense correspondence. Vis Comput 39, 6625–6638 (2023). https://doi.org/10.1007/s00371-022-02752-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02752-3

Keywords

Search

Navigation