Skip to main content
Springer Nature Link
Log in

Local features and manifold ranking coupled method for sketch-based 3D model retrieval

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

3D model retrieval can benefit many downstream virtual reality applications. In this paper, we propose a new sketch-based 3D model retrieval framework by coupling local features and manifold ranking. At technical fronts, we exploit spatial pyramids based local structures to facilitate the efficient construction of feature descriptors. Meanwhile, we propose an improved manifold ranking method, wherein all the categories between arbitrary model pairs will be taken into account. Since the smooth and detail-preserving line drawings of 3D model are important for sketch-based 3D model retrieval, the Difference of Gaussians (DoG) method is employed to extract the line drawings over the projected depth images of 3D model, and Bezier Curve is then adopted to further optimize the extracted line drawing. On that basis, we develop a 3D model retrieval engine to verify our method. We have conducted extensive experiments over various public benchmarks, and have made comprehensive comparisons with some state-of-the-art 3D retrieval methods. All the evaluation results based on the widely-used indicators prove the superiority of our method in accuracy, reliability, robustness, and versatility.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zeleznik R C, Herndon K P, Hughes J F. SKETCH: an interface for sketching 3D scenes. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. 1996, 163–170

    Google Scholar 

  2. Gao Y, Wang M, Tao D C, Ji R R, Dai Q H. 3-D Object retrieval and recognition with hypergraph analysis. IEEE Transactions on Image Processing, 2012, 21(9): 4290–4303

    Article  MathSciNet  MATH  Google Scholar 

  3. Walther D B, Chai B, Caddigan E, Beck DM, Li F F. Simple line drawings suffice for functional MRI decoding of natural scene categories. Proceedings of the National Academy of Sciences, 2011, 108(23): 9661–9666

    Article  Google Scholar 

  4. Eitz M, Hays J, Alexa M. How do humans sketch objects? ACM Transactions on Graphics, 2012, 31(4): 44

    Google Scholar 

  5. Funkhouser T, Min P, Kazhdan M, Chen J, Halderman A, Dobkin D, Jacobs D. A search engine for 3D models. ACM Transactions on Graphics, 2003, 22(1): 83–105

    Article  Google Scholar 

  6. Li B, Lu Y J, Ghumman A, Strylowski B, Gutierrez M, Sadiq S, Forster S, Feola N, Bugerin T. 3D Sketch-Based 3D model retrieval. In: Proceedings of the 5th ACM International Conference on Multimedia Retrieval. 2015, 555–558

    Chapter  Google Scholar 

  7. Liu X L, Huang L, Deng C, Tao D C. Query-Adaptive hash code ranking for large-scale multi-view visual search. IEEE Transactions on Image Processing, 2016, 25(10): 4514–4524

    Article  MathSciNet  Google Scholar 

  8. Liu X L, Du B W, Deng C, Liu M, Lang B. Structure sensitive hashing with adaptive product quantization. IEEE transactions on cybernetics, 2016, 46(10): 2252–2264

    Article  Google Scholar 

  9. DeCarlo D, Finkelstein A, Rusinkiewicz S, Santella. A Suggestive contours for conveying shape. ACM Transactions on Graphics, 2003, 22(3): 848–855

    Article  Google Scholar 

  10. Saavedra J M, Bustos B, Schreck T, Yoon S M, Scherer M. Sketchbased 3D model retrieval using keyshapes for global and local representation. In: Proceedings of the 5th Eurographics Conference on 3D Object Retrieval. 2012, 47–50

    Google Scholar 

  11. Yoon S, Scherer M, Schreck T, Kuijper A. A Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In: Proceedings of the 18th ACM International Conference on Multimedia. 2010, 193–200

    Google Scholar 

  12. Yoon SM, Kuijper A. Sketch-based 3D model retrieval using compressive sensing classification. Electronics Letters, 2011, 47(21): 1181–1183

    Article  Google Scholar 

  13. Judd T, Durand F, Adelson E. Apparent ridges for line drawing. ACM Transactions on Graphics, 2007, 26(3): 19

    Article  Google Scholar 

  14. Eitz M, Richter R, Boubekeur T, Hildebrand K Alexa M. Sketch-based shape retrieval. ACM Transactions on Graphics, 2012, 31(4): 1–10

    Google Scholar 

  15. Li B, Schreck T, Godil A, Alexa M, Boubekeur T, Bustos B, Chen J, Eitz M, Furuya T, Hildebrand K, Huang S, Johan H, Kuijper A, Ohbuchi R, Richter R, Saavedra J M, Scherer M, Yanagimachi T, Yoon G J, Yoon S M. Track: sketch-based 3D shape retrieval. In: Proceedings of the 5th Eurographics Conference on 3D Object Retrieval. 2012, 109–118

    Google Scholar 

  16. Shao T J, Xu W W, Yin K K, Wang J D, Zhou K, Guo B N. Discriminative sketch-based 3D model retrieval via robust shape matching. In: Proceedings of Comput Graph Forum. 2011

    Google Scholar 

  17. Li B, Lu Y, Li C, Godil A, Schreck T, Aono M, Burtscher M, Fu H, Furuya T, Johan H, Liu J, Ohbuchi R, Tatsuma A, Zou C. SHREC’14 track: extended large scale sketch-based 3D shape retrieval. In: Proceedings of Eurographics Workshop on 3D Object Retrieval. 2014

    Google Scholar 

  18. Li B, Lu Y, Godil A, Schreck T, Aono M, Johan H, Saavedra J M, Tashiro S. SHREC’13 track: scale sketch-based 3D shape retrieval. In: Proceedings of the 6th Eurographics Workshop on 3D Object Retrival. 2013, 89–96

    Google Scholar 

  19. Yoon S M, Yoon G, Schreck T. User-drawn sketch-based 3D object retrieval using sparse coding. Multimedia Tools and Applications. 2015, 74(13): 4707–4722

    Article  Google Scholar 

  20. Hu R, Wang T H, Collomosse J. A bag-of-regions approach to sketchbased image retrieval. In: Proceedings of the 18th IEEE International Conference on Image Processing. 2011, 3661–3664

    Google Scholar 

  21. Li B, Johan H. Sketch-based 3D model retrieval by incorporating 2D-3D alignment. Multimedia Tools and Applications, 2013, 65(3): 363–385

    Article  Google Scholar 

  22. Li B, Johan H. A 3D model feature for retrieval. In: Proceedings of MMM. 2010, 185–195

    Google Scholar 

  23. Belongie S, Malik J, Puzicha J. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4): 509–522

    Article  Google Scholar 

  24. Ohbuchi R, Furuya T. Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model. In: Proceedings of the 12th IEEE International Conference on Computer Vision. 2009, 63–70

    Google Scholar 

  25. Lowe D G. Distinctive Image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110

    Article  MathSciNet  Google Scholar 

  26. Wang F, Lin L F, Tang M. A new sketch-based 3D model retrieval approach by using global and local features. Graphical Models, 2014, 76(3): 128–139

    Article  Google Scholar 

  27. Wang F, Le K, Li Y. Sketch-based 3D shape retrieval using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1875–1883

    Google Scholar 

  28. Zhu F, Xie J, Fang Y. Learning cross-domain neural networks for sketch-based 3D shape retrieval. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016.

    Google Scholar 

  29. He J R, Li M J, Zhang H J, Tong H H, Zhang C S. Manifold-ranking based image retrieval. In: Proceedings of the 12th Annual ACM International Conference on Multimedia. 2004, 9–16

    Chapter  Google Scholar 

  30. Tong H H, He J R, Li M J, Ma W Y, Zhang H J, Zhang C S. Manifoldranking- based keyword propagation for image retrieval. EURASIP Journal on Applied Signal Processing, 2006, 2006: 190

    Google Scholar 

  31. Guan Z Y, Bu J J, Mei Q Z, Chen C, Wang C. Personalized tag recommendation using graph-based ranking on multi-type interrelated objects. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2009, 540–547

    Google Scholar 

  32. Agarwal S. Ranking on on graph data. In: Proceedings of the 23rd Internatioal Conference on Machine Learning. 2006, 25–32

    Google Scholar 

  33. Bu J J, Tan S L, Chen C, Wang C, Wu H, Zhang L J, He X F. Music recommendation by unified hypergraph: combining social media information and music content. In: Proceedings of the 18th ACM International Conference on Multimedia. 2010, 391–400

    Google Scholar 

  34. Furuya T, Ohbuchi R. Similarity metric learning for sketch-based 3D object retrieval. Multimedia Tools and Applications, 2015, 74(23): 10367–10392

    Article  Google Scholar 

  35. Liu X L, Lang B, Xu Y, Cheng B. Feature grouping and local soft match for mobile visual search. Pattern Recognition Letters, 2012, 33(3): 239–246

    Article  Google Scholar 

  36. Yang Y1, Nie F P, Xu D B, Luo J, Zhuang Y T, Pan Y H. Multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(4): 723–742

    Article  Google Scholar 

  37. Liu X L, Deng C, Lang B, Tao D C, Li X L. Query-Adaptive reciprocal hash tables for nearest neighbor search. IEEE Transactions on Image Processing, 2015, 25(2): 907–919

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Zhang Dongdong for his great help in experiments. This work was supported by the National Natural Science Foundation of China (Grant No. 61602324), the Scientific Research Project of Beijing Educational Committeen (KM201710028018), the open funding project of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (BUAA-VR-17KF-12) and Beijing Advanced Innovation Center for Imaging Technology (BAICIT-2016004).

Author information

Authors and Affiliations

  1. College of Information Engineering, Capital Normal University, Beijing, 100048, China

    Xiaohui Tan

  2. College of Information Science and Technology, Beijing Normal University, Beijing, 100875, China

    Yachun Fan

  3. School of Fashion, Beijing Institute of Fashion Technology, Beijing, 100028, China

    Ruiliang Guo

Authors
  1. Xiaohui Tan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yachun Fan
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ruiliang Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yachun Fan.

Additional information

Xiaohui Tan is a lecturer in Capital Normal University, China. Her main research interest is computer graphics and virtual reality.

Yachun Fan is an assistance professor in Beijing Normal University, China. Her main research interest is computer graphics and virtual reality.

Ruiliang Guo is an associate professor of Beijing Institute of Fashion Technology, China. His main interest is computer aided garment design and virtual reality.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tan, X., Fan, Y. & Guo, R. Local features and manifold ranking coupled method for sketch-based 3D model retrieval. Front. Comput. Sci. 12, 1000–1012 (2018). https://doi.org/10.1007/s11704-017-6595-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-017-6595-6

Keywords