Skip to main content

Advertisement

Springer Nature Link
Log in

3D Model Retrieval Using Bipartite Graph Matching Based on Attention

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this paper, we propose an attention-based bipartite graph 3D model retrieval algorithm, where many-to-many matching method, the weighted bipartite graph matching, is employed for comparison between two 3D models. Considering the panoramic views can donate the spatial and structural information, in this work, we use panoramic views to represent each 3D model. Attention mechanism is used to generate the weight of all views of each model. And then, we construct a weighted bipartite graph with the views of those models and the weight of each view. According to the bipartite graph, the matching result is used to measure the similarity between two 3D models. We experiment our method on ModelNet, NTU and ETH datasets, and the experimental results and comparison with other methods show the effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Ansary TF, Daoudi M, Vandeborre J-P (2007) A bayesian 3-d search engine using adaptive views clustering. IEEE Trans Multimed 9(1):78–88

    Article  Google Scholar 

  2. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

  3. Bai S, Bai X, Zhou Z, Zhang Z, Jan Latecki L (2016) Gift: a real-time and scalable 3d shape search engine. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5023–5032

  4. Bimbo AD, Pala P (2006) Content-based retrieval of 3d models. ACM Trans Multimed Comput Commun Appl (TOMM) 2(1):20–43

    Article  Google Scholar 

  5. Chen DY, Tian XP, Shen YT, Ming O (2003) On visual similarity based 3d model retrieval. Comput Graph Forum 22(3):223–232

    Article  Google Scholar 

  6. Chen J, Zhang H, He X, Nie L, Liu W, Chua T-S (2017) Attentive collaborative filtering: multimedia recommendation with item-and component-level attention. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp 335–344. ACM

  7. Chiotellis I, Triebel R, Windheuser T, Cremers D (2016) Non-rigid 3d shape retrieval via large margin nearest neighbor embedding. In: European conference on computer vision, pp 327–342. Springer

  8. Daras P, Axenopoulos A (2009) A compact multi-view descriptor for 3d object retrieval. In: International workshop on content-based multimedia indexing, pp 115–119

  9. Feng Y, Zhang Z, Zhao X, Ji R, Gao Y (2018) GVCNN: Group-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 264–272

  10. Gao Y, Tang J, Hong R, Yan S, Dai Q, Zhang N, Chua T-S (2012) Camera constraint-free view-based 3-d object retrieval. IEEE Trans Image Process 21(4):2269–2281

    Article  MathSciNet  Google Scholar 

  11. Gao Y, Wang M, Tao D, Ji R, Dai Q (2012) 3-d object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process 21(9):4290–4303

    Article  MathSciNet  Google Scholar 

  12. Garcia-Garcia A, Gomez-Donoso F, Garcia-Rodriguez J, Orts-Escolano S, Cazorla M, Azorin-Lopez J (2016) Pointnet: a 3d convolutional neural network for real-time object class recognition. In: 2016 International joint conference on neural networks (IJCNN), pp 1578–1584. IEEE

  13. Guo H, Wang J, Gao Y, Li J, Lu H (2016) Multi-view 3d object retrieval with deep embedding network. IEEE Trans Image Process Publ IEEE Signal Process Soc 25(12):5526–5537

    Article  MathSciNet  Google Scholar 

  14. He X, Chua T-S (2017) Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp 355–364. ACM

  15. Hilaga M, Shinagawa Y, Kohmura T, Kunii TL (2001) Topology matching for fully automatic similarity estimation of 3d shapes. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques, pp 203–212. ACM

  16. Kanezaki A, Matsushita Y, Nishida Y (2016) Rotationnet: joint object categorization and pose estimation using multiviews from unsupervised viewpoints. arXiv preprint arXiv:1603.06208

  17. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  18. Leibe B, Schiele B (2003) Analyzing appearance and contour based methods for object categorization. In: 2003 IEEE Computer society conference on computer vision and pattern recognition, 2003. Proceedings, vol 2, pp II–409. IEEE

  19. Leng B, Guo S, Changchun D, Zeng J, Xiong Z (2017) 3d object retrieval based on viewpoint segmentation. Multimed Syst 23(1):19–28

    Article  Google Scholar 

  20. Liu A, Wang Z, Nie W, Yuting S (2015) Graph-based characteristic view set extraction and matching for 3d model retrieval. Inf Sci 320:429–442

    Article  Google Scholar 

  21. Liu A-A, Nie W-Z, Gao Y, Yu-Ting S (2018) View-based 3-d model retrieval: a benchmark. IEEE Trans Cybern 48(3):916–928

    Google Scholar 

  22. Liu A-A, Nie W, Su Y (2019) 3d object retrieval based on multi-view latent variable model. IEEE Trans Circuits Syst Video Technol 29(3):868–880

    Article  Google Scholar 

  23. Maturana D, Scherer S (2015) Voxnet: a 3d convolutional neural network for real-time object recognition. In: 2015 IEEE/RSJ International conference on intelligent robots and systems (IROS), pp 922–928. IEEE

  24. Osada R, Funkhouser T, Chazelle B, Dobkin D (2002) Shape distributions. ACM Trans Graph (TOG) 21(4):807–832

    Article  MathSciNet  Google Scholar 

  25. Papadakis P, Pratikakis I, Theoharis T, Perantonis S (2010) Panorama: a 3d shape descriptor based on panoramic views for unsupervised 3d object retrieval. Int J Comput Vis 89(2–3):177–192

    Article  Google Scholar 

  26. Sfikas K, Pratikakis I, Theoharis T (2018) Ensemble of panorama-based convolutional neural networks for 3d model classification and retrieval. Comput Graph 71:208–218

    Article  Google Scholar 

  27. Sfikas K, Theoharis T, Pratikakis I (2017) Exploiting the panorama representation for convolutional neural network classification and retrieval. In: Eurographics workshop on 3D object retrieval, vol 8. The Eurographics Association

  28. Shi B, Bai S, Zhou Z, Bai X (2015) Deeppano: deep panoramic representation for 3-d shape recognition. IEEE Signal Process Lett 22(12):2339–2343

    Article  Google Scholar 

  29. Shu Z, Xin S, Huixia X, Kavan L, Wang P, Liu L (2016) 3d model classification via principal thickness images. Comput Aided Des 78:199–208

    Article  Google Scholar 

  30. Su H, Maji S, Kalogerakis E, Learned-Miller E (2015) Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE international conference on computer vision, pp 945–953

  31. Su H, Maji S, Kalogerakis E, Learned-Miller EG (2015) Multi-view convolutional neural networks for 3d shape recognition. ICCV, pp 945–953

  32. Vranic DV (2003) An improvement of rotation invariant 3d-shape based on functions on concentric spheres. In: 2003 International conference on image processing, 2003. ICIP 2003. Proceedings, vol 3, pp III–757. IEEE

  33. Wang D, Wang B, Zhao S, Yao H, Liu H (2017) View-based 3d object retrieval with discriminative views. Neurocomputing 252(C):58–66

    Article  Google Scholar 

  34. Nie W, Liu A-A, Gao Y, Su Y (2019) Hyper-clique graph matching and applications. IEEE Trans Circuits Syst Video Technol 29(6):1619–1630

    Article  Google Scholar 

  35. Wenshan H, Liu G-P, Zhou H (2013) Web-based 3-d control laboratory for remote real-time experimentation. IEEE Trans Ind Electron 60(10):4673–4682

    Article  Google Scholar 

  36. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1912–1920

  37. Zhang H, Kyaw Z, Chang S-F, Chua T-S (2017) Visual translation embedding network for visual relation detection. In: CVPR, vol 1, p 5

  38. Zhang H, Kyaw Z, Yu J, Chang S-F (2017) PPR-FCN: Weakly supervised visual relation detection via parallel pairwise R-FCN. arXiv preprint arXiv:1708.01956

  39. Zhang H, Niu Y, Chang S-F (2018) Grounding referring expressions in images by variational context. In: The IEEE conference on computer vision and pattern recognition

  40. Zhao S, Yao H, Yang Y, Zhang Y (2014) Affective image retrieval via multi-graph learning. In: Proceedings of the 22nd ACM international conference on multimedia, pp 1025–1028. ACM

Download references

Acknowledgements

This work was supported in part by the following projects: the National Natural Science Foundation of China through the Grants 61861014, the Guangxi Nature Science Fund (2016GXNSFAA380226), Guangxi Science and Technology Project (AC16380094, AA17204086), Guangxi Nature Science Fund Key Project (2016 GXNSFDA380031), and Guangxi University Science Research Project (ZD 2014146).

Author information

Authors and Affiliations

  1. College of Electronic Information and Automation, Guilin University of Aerospace Technology, Guilin, 541004, Guangxi, China

    Shanlin Sun, Yun Li, Yunfeng Xie, Zhicheng Tan & Xing Yao

  2. College of Information and Communication, Guilin University of Electronic Technology, Guilin, 541004, Guangxi, China

    Rongyao Zhang

Authors
  1. Shanlin Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yun Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yunfeng Xie
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Zhicheng Tan
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Xing Yao
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Rongyao Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yun Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, S., Li, Y., Xie, Y. et al. 3D Model Retrieval Using Bipartite Graph Matching Based on Attention. Neural Process Lett 52, 1043–1055 (2020). https://doi.org/10.1007/s11063-019-10155-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-019-10155-0

Keywords