Skip to main content
SpringerLink
Log in

Super resolution of single depth image based on multi-dictionary learning with edge feature regularization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Currently, the acquisition and application of depth images are attracting a lot of attention thanks to the rapid development of 3D video. However, the current depth cameras cannot obtain high quality depth images due to the limitations in the imaging system. In this paper, to address this issue, we propose a scheme for single depth image super resolution based on multi-dictionary learning with edge regularization model. In the training stage, we focus on the edge information that represents the structure of the image, and extract the edge part, the low frequency and high frequency parts from the high resolution depth image set respectively. After that, three dictionaries are learned for the three parts with the constraint of the same sparse representation. In the image synthesis stage, we employ an edge-preserving regularization model as a reconstruction constraint to preserve the sharp structure, and reconstruct the depth image via the dictionaries learned from the training stage. Experimental results show that our proposed method can achieve good results in edge preservation, and both PSNR and SSIM values of the reconstructed depth images are superior to the state-of-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Lowcomplexity single-image super-resolution based on nonnegative neighbor embedding. In: British machine vision conference

    Google Scholar 

  2. Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. Computer Vision and Pattern Recognition. CVPR 2005. IEEE Computer Society Conference. 2005:60–65 vol. 2

  3. Coates A, Ng AY (2012) Learning feature representations with K-Means. Neural networks: Tricks of the trade. Springer, Berlin Heidelberg, p 561–580

  4. Dong C, Chen CL, He K et al (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  5. Dong Y et al (2017) Depth map upsampling using joint edge-guided convolutional neural network for virtual view synthesizing. J Electronic Imaging 26(4):043004

    Article  Google Scholar 

  6. Elad M, Aharon M (2006) Image Denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745

    Article  MathSciNet  Google Scholar 

  7. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65

    Article  Google Scholar 

  8. Gui Z, Liu Y, He J (2012) PML algorithm for positron emission tomography combined with nonlocal fuzzy anisotropic diffusion filtering. IEEE Trans Nucl Sci 59(5):1984–1989

    Article  Google Scholar 

  9. Ha S, Mueller K (2015) Low dose CT image restoration using a database of image patches. J Phys Med Biol 60(2):869–882

    Article  Google Scholar 

  10. Han JW, Kim JH, Cheon SH et al (2010) A novel image interpolation method using the bilateral filter. IEEE Trans Consum Electron 56(1):175–181

    Article  Google Scholar 

  11. Keys RG (2003) Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process 29(6):1153–1160

    Article  MathSciNet  Google Scholar 

  12. Köhler T, Maier A, Christlein V (2015) Binarization driven blind Deconvolution for document image restoration. German conference on. Pattern Recogn:91–102

  13. Lee C, Eden M, Unser M (1998) High quality image resizing using oblique projection operators. IEEE Trans Image Process 7(5):679–692

    Article  Google Scholar 

  14. Li S, Jiang H, Pang W (2017) Joint multiple fully connected convolutional neural network with extreme learning machine for hepatocellular carcinoma nuclei grading. Comput Biol Med 84:145–167

    Article  Google Scholar 

  15. Li Y, Xue T, Sun L et al (2012) Joint Example-Based Depth Map Super-Resolution. IEEE International Conference on Multimedia and Expo, p 152–157

  16. Li C, Yung NHC, Sun X et al (2017) Human arm pose modeling with learned features using joint convolutional neural network. Mach Vis Appl 28(1–2):1–14

    Article  Google Scholar 

  17. Pati YC, Rezaiifar R, Krishnaprasad PS (2002) Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. Signals, Systems and Computers, 1993. 1993 Conference Record of The Twenty-Seventh Asilomar Conference on. IEEE, 40–44 vol.1

  18. Peleg T, Elad M (2014) A statistical prediction model based on sparse representations for single image super-resolution. IEEE Press

  19. Protter M, Elad M, Takeda H et al (2008) Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans Image Process 18(1):36–51

    Article  MathSciNet  Google Scholar 

  20. RaduTimofte, Vincent De Smet (2014) Luc Van Gool:A+: adjusted anchored neighborhood regression for fast super-resolution, ACCV

  21. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  22. Santosh KC, Vajda S, Antani S (2016) Edge map analysis in chest X-rays for automatic pulmonary abnormality screening. Int J Comput Assist Radiol Surg 11(9):1637–1646

  23. Santosh K C , Aafaque A , Antani S (2017) Line segment-based stitched multipanel figure separation for effective biomedical CBIR. IJPRAI 31(6):1–18

  24. Aafaque A, Santosh KCA (2016) Automatic compound figure separation in scientific articles: a study of edge map and its role for stitched panel boundary detection. RTIP2R: 319–332

  25. Santosh KC, Vajda S, Antani S (2015) Automatic pulmonary abnormality screening using thoracic edge map CBMS, 360–361

  26. R. Timofte, V. De, and L. V. Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 1920–1927

  27. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color image. computer vision, 1998. Sixth International Conference on IEEE:839–846

  28. Tronicke J, Böniger U (2013) Steering kernel regression: An adaptive denoising tool to process GPR data. IEEE International Workshop on Advanced Ground Penetrating Radar, p 1–4

  29. Wang R (2016) Edge Deterction using convolutional neural network. international symposium on neural networks. Springer International Publishing

  30. Xie J, Feris RS, Sun MT (2015) Edge guided single depth image super resolution. IEEE International Conference on Image Processing, p 3773–37777

  31. Xiong Z, Xu D, Sun X et al (2013) Example-based super-resolution with soft information and decision. IEEE Trans Multimedia 15(6):1458–1465

    Article  Google Scholar 

  32. Yang Y, Wang Z (2011) A new image super-resolution method in the wavelet domain. International conference on image & graphics. IEEE:163–167

  33. Yang Y, Wang Z (2012) Range image super-resolution via guided image filter. International conference on internet multimedia computing and service. ACM:200–203

  34. Yang S, Wang Z, Zhang L, Wang M (2014) Dual-geometric neighbor embedding for image super resolution with sparse tensor. IEEE Trans Image Process 23(7):2793–2803

    Article  MathSciNet  Google Scholar 

  35. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  36. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. International Conference on Curves and Surfaces. Springer-Verlag, 711–730

  37. Zhang J, Chen Z, Xiong R, et al. Image super-resolution via dual-dictionary learning and sparse representation. IEEE International Symposium on Circuits and Systems. 2012:1688–1691.

  38. Zhang A, Jiang H, Ma L et al (2016) A Shearlet-based algorithm for quantum noise removal in low-doseCT images, SPIE Medical Imaging. International Society for Optics and Photonics, p 97843O-97843O-7

  39. Zhang Y, Zhang Y, Zhang J et al (2015) Single image super-resolution via iterative collaborative representation. Proceedings, Part II, of the 16th Pacific-Rim Conference on Advances in Multimedia Information Processing -- PCM 2015 - Volume 9315. Springer-Verlag New York, Inc., 63–73

  40. Zhang Y, Zhang Y, Zhang J et al (2016) CCR: clustering and collaborative representation for fast single image super-resolution. IEEE Trans Multimedia 18(3):405–417

    Article  Google Scholar 

  41. Zhao L, Bai H, Liang J et al (2017) Single depth image super-resolution with multiple residual dictionary learning and refinement. IEEE International Conference on Multimedia and Expo, p 739–744

  42. Zheng H, Bouzerdoum A, Phung SL (2013) Depth image super-resolution using multi-dictionary sparse representation. IEEE International Conference on Image Processing, p 957–961

  43. Zohora FT (2017) Foreign circular element detection in Chest X-Rays for Effective Automated Pulmonary Abnormality Screening. IJCVIP 7(2):36–49

Download references

Acknowledgements

This work has been supported in part by National Natural Science Foundation of China (No.61672373 and No.61501315), Scientific and Technological Innovation Team of Shanxi Province (No. 201705D131025), Key Innovation Team of Shanxi 1331 Project(2017015), Collaborative Innovation Center of Internet+3D Printing in Shanxi Province(201708), The Program of “One hundred Talented People” of Shanxi Province, Shanxi Province Science Foundation for Youths (201701D221106); Taiyuan University of Science and Technology doctoral promoter (20162044). The authors thank the anonymous reviewers for their valuable comments and suggestions to improve the quality of this paper.

Author information

Authors and Affiliations

  1. Institute of Digital Multimedia and Communication, Taiyuan University of Science and Technology, Taiyuan, 030024, China

    Sihan Li, Anhong Wang, Hong Shangguan, Yingchun Wu, Donghong Li & Youcheng Wu

  2. Engineering Science, Simon Fraser University, Burnaby, BC, V5A 1S6, Canada

    Jie Liang

Authors
  1. Sihan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anhong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Shangguan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yingchun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Donghong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Youcheng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jie Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anhong Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, S., Wang, A., Shangguan, H. et al. Super resolution of single depth image based on multi-dictionary learning with edge feature regularization. Multimed Tools Appl 79, 34813–34834 (2020). https://doi.org/10.1007/s11042-019-08500-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08500-5

Keywords

Search

Navigation