Skip to main content
SpringerLink
Log in

Orthogonal integral transform for 3D shape recognition with few examples

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

3D shape recognition with few examples is crucial for applications involving 3D scenes, but typical methods based on surface and view suffer the failure to describe the interior and exterior features uniformly. Thus, we propose 3D orthogonal integral transform (OIT). OIT is composed of three individual integrals over a group of three orthogonal planes rotating to cover all orientations by which the volumetric shape is bisected in integrals. OIT offers the following advantages: (1) It describes a 3D shape structurally from interior to exterior uniformly, which brings about discriminative shape characteristics; and (2) the shape descriptor built on OIT is invariant with respect to translation, scaling and rotation. Furthermore, a fine-grained 3D model dataset (FGModele40) is built on ModelNet40. Experiments show that OIT can provide both discriminative and robust descriptors for 3D shape recognition with few examples. Our proposed OIT outperforms typical state-of-the-art benchmarks evaluated by the protein shape retrieval contest; additionally, it also surpasses other typical deep learning models with respect to the task of 3D shape recognition with few examples on FGModele40.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data that support the findings of this study has been available at https://github.com/lcd21/OIT/tree/master/FGModele40. More details may be available upon reasonable request by contacting with the corresponding author.

References

  1. Guo, Y., Bennamoun, M., Sohel, F.A., Lu, M., Wan, J., Kwok, N.M.: A comprehensive performance evaluation of 3d local feature descriptors. Int. J. Comput. Vis. 116(1), 66–89 (2016)

    Article  MathSciNet  Google Scholar 

  2. Langenfeld, F., Axenopoulos, A., Chatzitofis, A., Craciun, D., Daras, P., Du, B., Giachetti, A, kun Lai, Y., Li, H., Li, Y., Masoumi, M., Peng, Y., Rosin, P. L., Sirugue, J., Sun, L., Thermos, S., Toews, M., Wei, Y., Wu, Y., Zhai, Y., Zhao, T., Zheng, Y., Montes, M.: Shrec 2018 - protein shape retrieval, In: Eurographics conference on 3d object retrieval (2018)

  3. Daras, P., Zarpalas, D., Tzovaras, D., Strintzis, M. G.: Shape matching using the 3d radon transform, In: International symposium on 3D data processing, visualization and transmission, pp. 953–960 (2004)

  4. Daras, P., Zarpalas, D., Tzovaras, D., Strintzis, M.G.: Efficient 3-d model search and retrieval using generalized 3-d radon transforms. IEEE Trans. Multim. 8(1), 101–114 (2006)

    Article  Google Scholar 

  5. Tabbone, S., Wendling, L., Salmon, J.: A new shape descriptor defined on the radon transform. Comput. Vis. Image Underst. 102(1), 42–51 (2006)

    Article  Google Scholar 

  6. Averbuch, A., Shkolnisky, Y.: 3d fourier based discrete radon transform. Appl. Comput. Harm. Anal. 15(1), 33–69 (2003)

    Article  MathSciNet  Google Scholar 

  7. Axenopoulos, A., Rafailidis, D., Papadopoulos, G.T., Houstis, E.N., Daras, P.: Similarity search of flexible 3d molecules combining local and global shape descriptors. IEEE Trans. Comput. Biology Bioinform. 13(5), 954–970 (2016)

    Article  Google Scholar 

  8. Sit, A., Shin, W., Kihara, D.: Three-dimensional krawtchouk descriptors for protein local surface shape comparison. Patt. Recognit. 93, 534–545 (2019)

    Article  Google Scholar 

  9. Craciun, D., Levieux, G., Montès, M.: Shape similarity system driven by digital elevation models for non-rigid shape retrieval, In: I. Pratikakis, F. Dupont, M. Ovsjanikov (Eds.), Eurographics Workshop on 3D Object Retrieval (2017)

  10. Johnson, A.E., Hebert, M.: Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Trans. Patt. Anal. Mach. Intell. 21(5), 433–449 (1999)

    Article  Google Scholar 

  11. Guo, Y., Sohel, F.A., Bennamoun, M., Lu, M., Wan, J.: Rotational projection statistics for 3d local surface description and object recognition. Int. J. Comput. Vision 105(1), 63–86 (2013)

    Article  MathSciNet  Google Scholar 

  12. Masoumi, M., Rezaei, M., Ben, H.A.: Global spectral graph wavelet signature for surface analysis of carpal bones. Phys. Med. Biol. 63(3), 34–35 (2015)

    Google Scholar 

  13. Giachetti, A., Lovato, C.: Radial symmetry detection and shape characterization with the multiscale area projection transform. Comput. Graph. Forum 35(5), 1669–1678 (2012)

    Article  Google Scholar 

  14. Aubry, M., Schlickewei, U., Cremers, D.: The wave kernel signature: a quantum mechanical approach to shape analysis, In: IEEE International conference on computer vision workshops, pp. 1626–1633 (2011)

  15. Mirloo, M., Ebrahimnezhad, H.: Non-rigid 3d object retrieval using directional graph representation of wave kernel signature. Multim. Tools Appl. 77(6), 6987–7011 (2018)

    Article  Google Scholar 

  16. Cosmo, L., Minello, G., Bronstein, M. M., Rossi, L., Torsello, A.: The average mixing kernel signature, In: European Conference on computer vision, Vol. 12365, pp. 1–17 (2020)

  17. Zhang, D., Wu, Z., Wang, X., Lv, C., Zhou, M.: 3d non-rigid shape similarity measure based on fréchet distance between spectral distance distribution curve. Multim. Tools Appl. 80(1), 615–640 (2021)

    Article  Google Scholar 

  18. Liu, Y., Ye, Q., Wang, L., Peng, J.: Learning structural motif representations for efficient protein structure search. Bioinform 34(17), 1773–1780 (2018)

    Article  Google Scholar 

  19. Suryanto, C.H., Saigo, H., Fukui, K.: Structural class classification of 3d protein structure based on multi-view 2d images. IEEE Trans. Comput. Biol. Bioinform. 15(1), 286–299 (2018)

    Article  Google Scholar 

  20. Roth, H.R., Lu, L., Liu, J., Yao, J., Seff, A., Cherry, K.M., Kim, L., Summers, R.M.: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5), 1170–1181 (2016)

    Article  Google Scholar 

  21. Setio, A.A.A., Ciompi, F., Litjens, G.J.S., Gerke, P.K., Jacobs, C., van Riel, S.J., Wille, M.M.W., Naqibullah, M., Sánchez, C.I., van Ginneken, B.: Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans. Med. Imag. 35(5), 1160–1169 (2016)

    Article  Google Scholar 

  22. Nardelli, P., Jimenez-Carretero, D., Bermejo-Peláez, D., Washko, G.R., Rahaghi, F.N., Ledesma-Carbayo, M.J., Estépar, R.S.J.: Pulmonary artery-vein classification in CT images using deep learning. Trans. Med. Imag. 37(11), 2428–2440 (2018)

    Article  Google Scholar 

  23. Xie, Y., Xia, Y., Zhang, J., Song, Y., Feng, D., Fulham, M., Cai, W.: Knowledge-based collaborative deep learning for benign-malignant lung nodule classification on chest CT. IEEE Trans. Medical Imag. 38(4), 991–1004 (2019)

    Article  Google Scholar 

  24. Zhang, D., Wu, Z., Wang, X., Lv, C., Liu, N.: 3d skull and face similarity measurements based on a harmonic wave kernel signature. Vis. Comput. 37(4), 749–764 (2021)

    Article  Google Scholar 

  25. Ganapathi, I.I., Ali, S.S., Prakash, S.: Geometric statistics-based descriptor for 3d ear recognition. Vis. Comput. 36(1), 161–173 (2020)

    Article  Google Scholar 

  26. Bahroun, S., Abed, R., Zagrouba, E.: Deep 3d-lbp: Cnn-based fusion of shape modeling and texture descriptors for accurate face recognition. Vis. Comput. 39(1), 239–254 (2023)

    Article  Google Scholar 

  27. Peng, Z., Li, Z., Zhang, J., Li, Y., Qi, G.-J., Tang, J.: Few-shot image recognition with knowledge transfer, In: IEEE International conference on computer vision, pp. 441–449 (2019)

  28. Li, Z., Tang, J., Mei, T.: Deep collaborative embedding for social image understanding. IEEE Trans. Patt. Anal. Mach. Intell. 41(9), 2070–2083 (2019)

    Article  Google Scholar 

  29. Li, Z., Tang, J., Zhang, L., Yang, J.: Weakly-supervised semantic guided hashing for social image retrieval. Int. J. Comput. Vis. 128(8), 2265–2278 (2020)

    Article  MathSciNet  Google Scholar 

  30. Lin, C., Xiong, S.: Controllable face editing for video reconstruction in human digital twins. Image Vis. Comput. 125, 104517 (2022)

    Article  Google Scholar 

  31. Lin, C., Xiong, S., Chen, Y.: Mutual information maximizing GAN inversion for real face with identity preservation. J. Vis. Commun. Image Represent. 87, 103566 (2022)

    Article  Google Scholar 

  32. Lin, C., Xiong, S., Lu, X.: Disentangled face editing via individual walk in personalized facial semantic field. Vis. Comput. (2022). https://doi.org/10.1007/s00371-022-02708-7

    Article  Google Scholar 

  33. Maturana, D., Scherer, S. A.: Voxnet: a 3d convolutional neural network for real-time object recognition, In: International conference on intelligent robots and systems, 2015, pp. 922–928 (2015)

  34. Qi, C. R., Su, H., Mo, K., Guibas, L. J.: Pointnet: deep learning on point sets for 3d classification and segmentation, In: Conference on computer vision and pattern recognition, pp. 77–85 (2017)

  35. Shao, T., Yang, Y., Weng, Y., Hou, Q., Zhou, K.: H-cnn: spatial hashing based cnn for 3d shape analysis, IEEE Transactions on visualization and computer graphics, 1–12 (2018)

  36. Li, Y., Pirk, S., Su, H., Qi, C. R., Guibas, L. J.: FPNN: field probing neural networks for 3d data, In: Conference on neural information processing systems, pp. 307–315 (2016)

  37. Wu, J., Zhang, C., Xue, T., Freeman, B., Tenenbaum, J.: Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling, In: Advances in neural information processing, pp. 82–90 (2016)

  38. Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E. G.: Multi-view convolutional neural networks for 3d shape recognition, In: IEEE International conference on computer vision, pp. 945–953 (2015)

  39. He, X., Bai, S., Chu, J., Bai, X.: An improved multi-view convolutional neural network for 3d object retrieval. IEEE Trans. Image Process. 29, 7917–7930 (2020)

    Article  Google Scholar 

  40. Qi, C. R., Su, H., Nießner, M., Dai, A., Yan, M., Guibas, L. J.: Volumetric and multi-view cnns for object classification on 3d data, In: IEEE Conference on computer vision and pattern recognition, pp. 5648–5656 (2016)

  41. Bai, S., Bai, X., Zhou, Z., Zhang, Z., Latecki, L. J.: GIFT: a real-time and scalable 3d shape search engine, In: IEEE Conference on computer vision and pattern recognition, pp. 5023–5032 (2016)

  42. Kanezaki, A., Matsushita, Y., Nishida, Y.: Rotationnet for joint object categorization and unsupervised pose estimation from multi-view images. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 269–283 (2021)

    Article  Google Scholar 

  43. Chu, H., Le, C., Wang, R., Li, X., Ma, H.: Learning representative viewpoints in 3d shape recognition. Vis. Comput. 38(11), 3703–3718 (2022)

    Article  Google Scholar 

  44. Ioannidou, A., Chatzilari, E., Nikolopoulos, S., Kompatsiaris, I.: Deep learning advances in computer vision with 3d data: a survey. ACM Comput. Surv. 50(2), 2001–2038 (2017)

    Google Scholar 

  45. Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3d shapenets: a deep representation for volumetric shapes, In: Conference on computer vision and pattern recognition, pp. 1912–1920 (2015)

  46. Liu, X., Huang, H., Wang, W., Zhou, J.: Multi-view 3d shape style transformation. Vis. Comput. 38(2), 669–684 (2022)

    Article  Google Scholar 

  47. Lei, H., Akhtar, N., Mian, A.: Spherical kernel for efficient graph convolution on 3d point clouds. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3664–3680 (2021)

    Article  Google Scholar 

  48. Han, L., Piao, J., Tong, Y., Yu, B., Lan, P.: Deep learning for non-rigid 3d shape classification based on informative images. Multim. Tools Appl. 80(1), 973–992 (2021)

    Article  Google Scholar 

  49. Shilane, P., Min, P., Kazhdan, M. M., Funkhouser, T. A.: The princeton shape benchmark, In: International conference on shape modeling and applications, pp. 167–178 (2004)

  50. Wang, B., Gao, Y.: Structure integral transform versus radon transform: a 2d mathematical tool for invariant shape recognition. IEEE Trans. Image Process. 25(12), 5635–5648 (2016)

    Article  MathSciNet  Google Scholar 

  51. Li, H., Sun, L., Wu, X., Cai, Q.: Scale-invariant wave kernel signature for non-rigid 3d shape retrieval, In: IEEE International conference on big data and smart computing, pp. 448–454 (2018)

  52. Benhabiles, H., Hammoudi, K., Windal, F., Melkemi, M., Cabani, A.: A transfer learning exploited for indexing protein structures from 3d point clouds, In: Processing and analysis of biomedical information, pp. 82–89 (2019)

  53. Langenfeld, F., Peng, Y., Lai, Y., Rosin, P.L., Aderinwale, T., Terashi, G., Christoffer, C., Kihara, D., Benhabiles, H., Hammoudi, K., Cabani, A., Windal, F., Melkemi, M., Giachetti, A., Mylonas, S.K., Axenopoulos, A., Daras, P., Otu, E., Montès, M.: SHREC 2020: multi-domain protein shape retrieval challenge. Comput. Graph. 91, 189–198 (2020)

    Article  Google Scholar 

  54. Said, S., Le Bihan, N., Sangwine, S.J.: Fast complexified quaternion fourier transform. IEEE Trans. Signal Process. 56(4), 1522–1531 (2008)

    Article  MathSciNet  Google Scholar 

  55. Berman, H.M., Westbrook, J.D., Feng, Z., Gilliland, G., Bhat, T.N., Weissig, H., Shindyalov, I.N., Bourne, P.E.: The protein data bank. Nucl. Acids Res. 28(1), 235–242 (2016)

    Article  Google Scholar 

  56. Fox, N.K., Brenner, S.E., Chandonia, J.: Scope: structural classification of proteins - extended, integrating scop and astral data and classification of new structures. Nucl. Acids Res. 42(1), 304–309 (2014)

    Article  Google Scholar 

Download references

Acknowledgements

This work was in part supported by NSFC (Grant No. 62176194, Grant No.62101393), the Major project of IoV (Grant No. 2020AAA001), Sanya Science and Education Innovation Park of Wuhan University of Technology (Grant No. 2021KF0031), CSTC (Grant No. cstc2021jcyj-msxmX1148) and the Open Project of Wuhan University of Technology Chongqing Research Institute (ZL2021-6).

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Guilin University Of Electronic Technology, Guilin, 541004, China

    Chengde Lin

  2. School of Mathematics and Computational Sciences, Wuyi University, Jiangmen, 529020, China

    Peng Wang

  3. School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, 430070, China

    Shengwu Xiong & Ruyi Chen

Authors
  1. Chengde Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shengwu Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruyi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

CL: Conceptualization, Methodology, Software, Writing-Original Draft, Formal analysis, Writing-Review & Editing. PW: Validation, Resources, Writing-Review & Editing. SX: Conceptualization, Methodology, Writing-Review & Editing, Project administration. RC: Validation, Resources, Writing-Review & Editing.

Corresponding author

Correspondence to Chengde Lin.

Ethics declarations

Conflict of interest

The authors declare that they have no known conflict of interest or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 56 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, C., Wang, P., Xiong, S. et al. Orthogonal integral transform for 3D shape recognition with few examples. Vis Comput 40, 3271–3284 (2024). https://doi.org/10.1007/s00371-023-03030-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-03030-6

Keywords

Search

Navigation