Skip to main content
Springer Nature Link
Log in

Deep convolution network for surveillance records super-resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The aim of image super resolution (SR) is to recover low resolution (LR) input image or video to a visually desirable high-resolution (HR) one. The task of identifying an object in surveillance records is interesting, yet challenging due to the low resolution of the video. This paper, proposed a deep learning method for resolution recovery, the low-resolution objects and points in the surveillance records are up-sampled using a deep Convolutional Neural Network (CNN) to avoid problems of image boundary the data padded with zeros. The network is trained and tested on two surveillance datasets. Dissimilar to the outdated methods which operate components individually, our model performs combined optimization for all the layers. The proposed CNN model has a lightweight structure and minimal data pre-processing and computation cost. Testing our model and comparing with advanced techniques, we observed promising results. The code is accessible at https://github.com/Mzareapoor/Super-resolution

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Al-Najjar YAY, Soong DDC (2012) Comparison of image quality assessment: PSNR, HVS, SSIM, UIQI. International Journal of Scientific and Engineering Research 3(8):1–5

    Google Scholar 

  2. Bengio Y, Goodfellow IJ, Courville A (2015) Deep learning. Book in preparation for MIT Press. 2015

  3. Cai D, Chen K, Qian Y, Kämäräinen JK (2017) Convolutional low-resolution fine-grained classification. Pattern recognition letters. https://doi.org/10.1016/j.patrec.2017.10.020

  4. Cui Z, Chang H, Shan S, Zhong B, Chen X (2014) Deep network cascade for image super-resolution. In: European Conference on Computer Vision, pp 49–64

  5. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: ECCV

  6. Dong C, Loy CC, He K, Tang X (2015) Image super resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence

  7. Fu Z, Li Z, Ding L, Nguyen T (2014) Translation invariance-based super resolution method for mixed resolution multiview video. In: ICIP

  8. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. IEEE International Conference on Computer Vision, pp 349–356

  9. Grgic M, Delac K, Grgic S (2011) SCface - surveillance cameras face database. Multimedia Tools and Applications 51(3):863–879

    Article  Google Scholar 

  10. He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Proc. Eur. Conf. Computer vision, pp 346–361

  11. Huang J-B, Singh A, Ahuja N (2015) Single image super resolution using transformed self-exemplars. In: CVPR

  12. Hung EM, Dorea CC, Garcia DC, Queiroz RL (2010) Transform-domain super resolution for multiview images using depth information. In: EUSIPCO

  13. Irani M, Peleg S (1991) Improving resolution by image registration. CVGIP: Graphical models and image processing 53(3):231–239, 1991

    Google Scholar 

  14. Jain AK, Nguyen TQ (2013) Video super resolution for mixed resolution stereo. In: ICIP

  15. Jin Z, Tillo T, Yao C, Xiao J, Zhao Y (2015) Virtual view assisted video super-resolution and enhancement. IEEE transactions on circuits and Systems for Video Technology, pp 467–478

  16. Joachimiak M, Aflaki P, Hannuksela MM, Gabbouj M (2014) Evaluation of depth-based super resolution on compressed mixed resolution 3d video. In: ACCV

  17. Kim KI, Kown Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133

    Article  Google Scholar 

  18. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. CoRR, abs/1412.6980

  19. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In Proc. Adv Neural Inf Process Syst, pp 1097–1105

  20. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradientbased learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  21. Liao R, Tao X, Li R, Ma Z, Jia J (2015) Video superresolution via deep draft-ensemble learning. IEEE International Conference on Computer Vision, pp 531–539

  22. Liu C, Sum D (2014) On bayesian adaptive video super resolution. IEEE Trans Pattern Anal Mach Intell 36(2):346–360

    Article  Google Scholar 

  23. Marco Bevilacqua CG, Roumy A, Morel M-LA (2012) Low-complexitysingle-imagesuper-resolutionbased on nonnegative neighbor embedding. In: BMVC

  24. Na Z, Liao R, Tao X, Xu L, Jia J, Wu E (2015) Handling motion blur in multi-frame super-resolution. CVPR

  25. Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proc. Int. Conf. Mach Learn, pp 807–814

  26. Ouyang W, Wang X, Zeng X, Qiu S, Luo P, Tian Y, Li H, Yang S, Wang Z, Loy C-C, Tang X (2015) Deepid-net: deformable deep convolutional neural networks for object detection. In: Proc. IEEE Conf. Comput. Vis. Pattern Recogn. pp 2403–2412

  27. Razvan P, Tomas M, Yoshua B (2013) On the difficulty of training recurrent neural networks. ICML

  28. Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests. In: CVPR

  29. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLR. arXiv:1409.1556v6

  30. Song X, Dai Y, Qin X (2016) Deep depth super-resolution: learning depth super-resolution using deep convolutional neural network. Computer Vision – ACCV 2016 pp 360–376

  31. Stelmach L, Tom WJ, Meegan D, Vincent A (2000) Stereo image quality: effects of mixed spatio-temporal resolution. IEEE Transactions on Circuits and Systems for Video Technology 10(2):188–193

    Article  Google Scholar 

  32. Sun Y, Chen Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. In: Proc. Adv Neural Inf Process Syst, pp 1988–1996

  33. Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. Proceedings of the 30th international conference on Mach Learn, pp 1139–1147

  34. Timofte R, Smet VD, Gool LV (2013) Anchored neighborhood regression for fast example-based super-resolution. IEEE International Conference on Computer Vision, pp 1920–1927

  35. Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: CVPR

  36. Wong Y, Chen S, Mau S, Sanderson C, Lovell BC (2011) Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition. IEEE Biometrics Workshop, Computer Vision and Pattern Recognition (CVPR), pp 81–88

  37. Xie Y, Xiao J, Tillo T, Wei Y, Zhao Y (2016) 3D video super resolution using fully convolutional neural networks. IEEE International Conference on Multimedia and Expo (ICME)

  38. Yang J, Wright J, Huang T, Ma Y (2010) Image super resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  39. Yang C-Y, Ma C, Yang MH (2014) Single-image super resolution: a benchmark. European Conference on Computer Vision, pp 372–386

  40. Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: Curves and Surfaces, pp 711–730

  41. Zhang N, Donahue J, Girshick R, Darrell T (2014) Part-based RCNNs for fine-grained category detection. In: Proc. Eur. Conf. Comput. Vis., pp 834–849

  42. Zhao Y, Wang R, Dong W, Jia W, Yang J, Liu X, Gao W (2017) GUN: Gradual Upsampling Network for single image super-resolution. Computer Vision and Pattern Recognition (CVPR). arXiv:1703.04244

Download references

Author information

Authors and Affiliations

  1. Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, China

    Pourya Shamsolmoali, Masoumeh Zareapoor & Jie Yang

  2. Advanced Scientific Computing Division, Euro-Mediterranean Centre on Climate Change, Lecce, Italy

    Pourya Shamsolmoali

  3. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Deepak Kumar Jain

  4. Jaypee University of Engineering and Technology, Guna, India

    Vinay Kumar Jain

Authors
  1. Pourya Shamsolmoali
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Masoumeh Zareapoor
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Deepak Kumar Jain
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Vinay Kumar Jain
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Jie Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Pourya Shamsolmoali or Jie Yang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shamsolmoali, P., Zareapoor, M., Jain, D.K. et al. Deep convolution network for surveillance records super-resolution. Multimed Tools Appl 78, 23815–23829 (2019). https://doi.org/10.1007/s11042-018-5915-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5915-7

Keywords