Skip to main content
Springer Nature Link
Log in

Free-hand sketches for 3D model retrieval using cascaded LSDA

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

This study proposes a novel cascaded 3D model retrieval framework for automatically and accurately fitting 3D models to user freehand 2D sketches (or object contours). The matching result provides the 3D geometry hypotheses of the 2D objects, and thus it is an important and fundamental part of several computer vision tasks, such as spatial layout estimation and scene understanding. However, existing matching approaches are still restricted by the corresponding point/line matching between the 2D image and the target 3D model. The limitation may make existing matching methods not online accessible in practice, and degrade their applicability. An alternative scenario is proposed to address this problem, in this paper. We use a retrieval system to collect a 3D-2D database offline, in which the 3D models are decomposed into 2D projected contours with respective to a limited number of different viewpoints. Our approach aims to enhance matching results by leveraging the extra database. Specifically, it optimizes a transformation from 3D objects to 2D contours, by which the target 3D object pose and categorization, to be recognized can be concisely estimated by entries in such database. More importantly, we demonstrate how to use a set of training images with limited pose variation to handle test images taken under uncontrolled view.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Alvarado C, Cani MP (2008) Sketch-based search and composition of 3D models. EUROGRAPHICS workshop on sketch-based interfaces and modeling

  2. Arbter K, Snyder WE, Burkhardt H, Hirzinger G (1990) Application of affine-Invariant Fourier descriptors to Recognition of 3-D Objects. IEEE Trans Pattern Anal Mach Intell 12:640–647

    Article  Google Scholar 

  3. Atanasov N, Sankaran B, Ny JL, Pappas GJ, Daniilidis K (2014) Nonmyopic view planning for active object classification and pose estimation. IEEE Trans Robot, pp 1078–1090

  4. Cai D, He X, Zhou K, Han J, Bao H (2007) Locality sensitive discriminant. Analysis. In Proceedings of IJCAI, pp 708–713

  5. Chen Y, Dana S, van de Panne M (2005) Sketch-based modeling of parameterized objects. ACM SIGGRAPH 2005 Sketches

  6. Ecker A, Kutulakos KN, Jepson AD (2007) Shape from planar curves: a linear escape from flatland. Comput Vis Pattern Recognit, pp 1–8

  7. Eggli L, Hsu C-Y, Briiderlin BD, Elbert G (1997) Inferring 3D models from freehand sketches and constraints. Comput Aided Des Solid Model 29(2):101–112

    Article  Google Scholar 

  8. Eitz M, Richter R, Boubekeur T, Hildebrand K, Alexa M (2012) Sketch-based shape retrieval. ACM Trans Graph

  9. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM, pp 391–395

  10. Hedau V, Hoiem D, Forsyth DA (2009) Recovering the spatial layout of cluttered rooms. ICCV, pp 1849–1856

  11. Hedau V, Hoiem D, Forsyth DA (2010) Thinking inside the box: using appearance models and context based on room geometry. ECCV, pp 224–237

  12. Huang Z, Fu H, Lau WH (2014) Data-driven segmentation and labeling of freehand sketches. ACM Trans Graph

  13. HyoJong S, Igarashi T (2007) Magic canvas: interactive design of a 3-D scene prototype from freehand sketches. Proceedings of Graphics Interface, pp 63–70

  14. Karpenko OA, Hughes JF (2006) SmoothSketch: 3D free-form shapes from complex sketches. ACM Trans Graph 25(3):589–598

    Article  Google Scholar 

  15. Khan SH, He X, Bennamoun M, Sohel FA, Togneri R (2015) Separating objects and clutter in indoor scenes. CVPR, pp 4603–4611

  16. Lakshman L, Ramana R (2000) Multi-scale discretization of shape contours. Vision geometry IX, pp 30–31

  17. Lee DC, Gupta A, Hebert M, Kanade T (2010) Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces. NIPS, pp 1288–1296

  18. Li B, Lu Y, Fares R (2013) Semantic sketch-based 3D model retrieval. IEEE International Conference on Multimedia and Expo (ICME)

  19. Owada S, Nielsen F, Okabz M, Igarashi T (2004) Volumetric illustration designing 3D models with internal textures. ACM Transactions on Graphics, Proceedings of ACM SIGGRAPH, pp 322–328

  20. Payet N, Todorovic S (2011) From contours to 3D object detection and pose estimation. ICCV, pp 983–990

  21. Safar M, Shahabi C, Sun X (2000) Image Retrieval by shape: a comparative study. Multimed Expo 1:141–144

    Article  Google Scholar 

  22. Tanase M, Veltkamp RC (2005) Part-based shape retrieval with relevance feedback. Multimedia and Expo, pp 936–939

  23. Warren J, Weimer H (2001) Subdivision methods for geometric design: a constructive approach. Morgan Kaufmann Publishers Inc

  24. Xu B, Chang W, Sheffer A, Bousseau A, McCrae J (2014) Karan Singh True2Form: 3D curve networks from 2D sketches via selective regularization. ACM Transactions on Graphics (TOG)

  25. Xu K, Chen K, Fu H, Sun WL, Hu SM (2013) Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models. ACM Trans Graph

  26. Zhenguo L, Jianzhuang L, Xiaoou T (2007) A closed-form solution to 3D reconstruction of piecewise planar objects from Single Image. Comput Vis Pattern Recognit, pp 1–6

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, Tamkang University, No.151, Yingzhuan Rd., Tamsui Dist., New Taipei City, Taiwan, 25137, Republic of China

    Chin-Ting Tu

  2. MediaTek Inc., Hsinchu, 30078, Republic of China

    Po-Chung Lin

  3. National Cheng Kung University, Tainan, 70101, Republic of China

    Jenn-Jier Lien

Authors
  1. Chin-Ting Tu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Po-Chung Lin
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jenn-Jier Lien
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Chin-Ting Tu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tu, CT., Lin, PC. & Lien, JJ. Free-hand sketches for 3D model retrieval using cascaded LSDA. Multimed Tools Appl 76, 11669–11685 (2017). https://doi.org/10.1007/s11042-016-3326-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3326-1

Keywords