Skip to main content
Springer Nature Link
Log in

Depth map Super-Resolution based on joint dictionary learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Although Time-of-Flight (ToF) camera can provide real-time depth information from a real scene, the resolution of depth map captured by ToF camera is rather limited compared to HD color cameras, and thus it cannot be directly used in 3D reconstruction. In order to handle this problem, this paper proposes a novel compressive sensing (CS) and dictionary learning based depth map super-resolution (SR) method, which transforms a low resolution depth map to a high resolution depth map. Different from previous depth map SR methods, this algorithm uses a joint dictionary learning method with both low and high resolution depth maps, and this method also builds a sparse vector classification method which is used in depth map SR. Experimental results show that the proposed method outperforms state-of-the-art methods for depth map super-resolution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. 3dv systems, z-cam, http://www.3dvsystems.com

  2. Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183

    Article  Google Scholar 

  3. Baraniuk RG (2007) Compressive sensing. IEEE Signal Proc Mag 24(4):118–120

    Article  Google Scholar 

  4. Candès EJ (2006) Compressive sampling. Proc Int Congr Math Madrid Spain 3:1433–1452

    Google Scholar 

  5. Canesta Inc (2006) Canestavision electronic perception development kit, http://www.canesta.com/

  6. Chang H, Yeung D, Xiong Y (2004) Super-resolution through neighbor embedding. IEEE Conference on Computer Vision and Pattern Recognition, 1:1–8

  7. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306

    Article  MATH  MathSciNet  Google Scholar 

  8. Freeman W, Jones T, Pasztor E (2002) Example-based superresolution. IEEE Comput Graph Appl 22(2):56–65

    Article  Google Scholar 

  9. Freeman W, Pasztor E, Carmichael O (2000) Learning lowlevel vision. Int J Comput Vis 40(1):25–47

    Article  MATH  Google Scholar 

  10. Gao X, Zhang K, Li X, Tao D (2012) Joint learning for single-image super-resolution via a coupled constraint. IEEE Trans Image Process 21(2):469–480

    Article  MathSciNet  Google Scholar 

  11. Geman S, Geman D (1984) Stochastic relaxation, gibbs distribution, and the bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6(4):721–741

    Article  MATH  Google Scholar 

  12. Glasner D, Bagon S, Irani M (2009) “Super-resolution from a single image” ICCV pp. 349–356

  13. Han Y, Wu F, Tian Q, Zhuang Y (2012) Image annotation by input-output structural grouping sparsity. IEEE Trans Image Process (IEEE T-IP) 21(6):3066–3079

    Article  MathSciNet  Google Scholar 

  14. Han Y, Yang Y, Ma Z, Shen H, Sebe N, Zhou X (2014) Image attribute adaptation. IEEE Trans Multimed (IEEE T-MM). doi:10.1109/TMM.2014.2306092

    Google Scholar 

  15. Kopf J, Cohen MF, Lischinski D, Uyttendaele M (2007) Joint bilateral upsampling. ACM Trans Graph 26(3):96

    Article  Google Scholar 

  16. PMD camcube (2009) http://www.pmdtec.com/

  17. Protter M, Elad M, Takeda H, Milanfar P (2009) Generaliizing the nonlocal-means to super-resolution resconstruction. IEEE Trans Image Process 18(1):36–51

    Article  MathSciNet  Google Scholar 

  18. Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  19. Scharstein D and Szeliski R (2002) Middlebury stereo evaluation-version 2, http://vision.middlebury.edu/stereo/eval

  20. Tang Y, Yuan Y, Yan P, Li X (2011) Single-image superresolution via local learning. Int J Mach Learn Cybern 6(9):15–23

    Article  Google Scholar 

  21. Tseng P, Yun S (2009) A coordinate gradient descent method for nonsmooth separable minimization. Math Program Ser B 117:387–423

    Article  MATH  MathSciNet  Google Scholar 

  22. Wang J, Zhu S, Gong Y (2009) Resolution enhancement based on learning the sparse association of image patches. Pattern Recogn Lett 31(1):1–10

    Article  Google Scholar 

  23. Wu F, Lu X, Zhang Y, Zhang Z, Yan S, Zhuang Y (2013) Cross-media semantic representation via Bi-directional learning to rank. Proc 2013 ACM Int Conf Multimed (ACM Multimedia, Full Paper) 877–886

  24. Wu F, Zhou Y, Yang Y, Siliang T, Zhang Y, Zhuang Y (2014) Sparse multi-modal hashing. IEEE Trans Multimed 16(2):427–439

    Article  Google Scholar 

  25. Xu Z, Schwarte R, Heinol H, Buxbaum B, Ringbeck T (1998) Smart pixel C photonic mixer device (pmd), M2VIP 1998 - Int Conf Mechatron Mach Vision Pract 259–264

  26. Yang J, Wright J, Huang T, Ma Y (2010) Image superresolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  27. Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (Grant No. 61271338), the National High Technology Research and Development Program (863) of China (Grant No. 2012AA011505), the Zhejiang Provincial Natural Science Foundation of China (Grant No. Q14F010020), and the Open Projects Program of National Laboratory of Pattern Recognition of China (Grant No. 201306308).

Author information

Authors and Affiliations

  1. Institute of Information and Communication Engineering, Zhejiang University, Hangzhou, 310027, 38 zheda road, China

    Li-Wei Liu, Liang-Hao Wang & Ming Zhang

Authors
  1. Li-Wei Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Liang-Hao Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ming Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Liang-Hao Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, LW., Wang, LH. & Zhang, M. Depth map Super-Resolution based on joint dictionary learning. Multimed Tools Appl 74, 467–477 (2015). https://doi.org/10.1007/s11042-014-2002-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2002-6

Keywords