Skip to main content
SpringerLink
Log in

Nighttime visual refinement techniques for surveillance video: a review

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video surveillance systems substitute manual efforts in various safety critic domains such as border area, assisted living, banking, service stations, and transportation. The multimedia-based surveillance system has a significant role in security and forensic systems because people tend to be easily convinced after observing voice, image, and video. Hence, these videos are strong evidence in the forensic investigation. However, most of the criminal activities such as ATM robbery and assassination are occur at nighttime because of the crime supporting dark environment. Many of the night surveillance systems in military, as well as commercial applications, are equipped with infrared and thermal based night vision systems. Its poor capability of texture and color interpretations are the major issues to ensure secure nighttime video monitoring. Specifically, visual refinements of nighttime surroundings and foreground objects provide a valuable assistance in the nighttime security system. In this scenario, it is highly recommended a review of the state-of-the-art nighttime visual refinement approaches. We conducted an extensive literature review and classified the nighttime visual refinement approaches into nighttime restoration and enhancement. This comparative literary analysis identified the research gap fields to explore future research directions in nighttime visual enhancement techniques. Finally, we discussed various open issues and future directions in the context enhancement based nighttime enhancement research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Al-Ameen Z (2019) Nighttime image enhancement using a new illumination boost algorithm. IET Image Process

  2. Benoit A, Caplier A, Durette B, Hérault J (2010) Using human visual system modeling for bio-inspired low level image processing. Comput Vis Image Underst 114(7):758–773

    Article  Google Scholar 

  3. Cai L, Qian J (2009) Night color image enhancement using fuzzy set. In: 2nd International congress on image and Signal Processing, 2009. CISP’09. IEEE, pp 1–4

  4. Celik T (2013) Spatio-temporal video contrast enhancement. IET Image Process 7(6):543–555

    Article  Google Scholar 

  5. Chen Y, Lin W, Zhang C, Chen Z, Xu N, Xie J (2013) Intra-and-inter-constraint-based video enhancement based on piecewise tone mapping. IEEE Trans Circuits Syst Video Technol 23(1):74–82

    Article  Google Scholar 

  6. Cheng H-Y, Yu C-C (2014) Nighttime traffic flow analysis for rain-drop tampered cameras. In: 2014 22nd International conference on pattern recognition (ICPR). IEEE, pp 714–719

  7. Chouhan R, Biswas PK, Jha RK (2015) Enhancement of low-contrast images by internal noise-induced fourier coefficient rooting. Signal Image Video Process 9(1):255–263

    Article  Google Scholar 

  8. Chouhan R, Jha RK, Biswas PK (2013) Enhancement of dark and low-contrast images using dynamic stochastic resonance. IET Image Process 7(2):174–184

    Article  MathSciNet  Google Scholar 

  9. Chouhan R, Jha RK, Biswas PK (2013) Noise-enhanced contrast stretching of dark images in svd-dwt domain. In: 2013 Annual IEEE India conference (INDICON). IEEE, pp 1–6

  10. Dong X, Li W, Wang G, Lu Y, Meng W, et al. (2011) An efficient and integrated algorithm for video enhancement in challenging lighting conditions. arXiv:1102.3328

  11. Dong X, Wang G, Pang Y, Li W, Wen J, Meng W, Lu Y (2011) Fast efficient algorithm for enhancement of low lighting video. In: 2011 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6

  12. Fan X, Wang L (2019) Image defogging approach based on incident light frequency, Multimed Tools Appl. [Online]. Available: https://doi.org/10.1007/s11042-018-7103-1

    Article  Google Scholar 

  13. Fu X, Zeng D, Huang Y, Liao Y, Ding X, Paisley J (2016) A fusion-based enhancing method for weakly illuminated images. Signal Process 129:82–96

    Article  Google Scholar 

  14. Honda H, Timofte R, Van Gool L (2015) Make my day-high-fidelity color denoising with near-infrared. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 82–90

  15. Hu J, Hu R, Wang Z, Gong Y, Duan M (2013) Kinect depth map based enhancement for low light surveillance image. In: 2013 20th IEEE International conference on image processing (ICIP). IEEE, pp 1090–1094

  16. Hu Y, Shang Y, Fu X, Ding H (2015) A low illumination video enhancement algorithm based on the atmospheric physical model. In: 2015 8th International congress on image and signal processing (CISP). IEEE, pp 119–124

  17. Ilie A, Raskar R, Yu J (2005) Gradient domain context enhancement for fixed cameras. Int J Pattern Recognit Artif Intell 19(04):533–549

    Article  Google Scholar 

  18. Jha RK, Chouhan R, Biswas PK, Aizawa K (2012) Internal noise-induced contrast enhancement of dark images. In: 2012 19th IEEE International conference on image processing (ICIP). IEEE, pp 973– 976

  19. Jiang X, Yao H, Liu D (2019) Nighttime image enhancement based on image decomposition. SIViP 13(1):189–197

    Article  Google Scholar 

  20. Jiang X, Yao H, Zhang S, Lu X, Zeng W (2013) Night video enhancement using improved dark channel prior. In: 2013 IEEE International conference on image processing. IEEE, pp 553–557

  21. Jung C, Yang Q, Sun T, Fu Q, Song H (2017) Low light image enhancement with dual-tree complex wavelet transform. J Vis Commun Image Represent 42:28–36

    Article  Google Scholar 

  22. Kim M, Park D, Han DK, Ko H (2015) A novel approach for denoising and enhancement of extremely low-light video. IEEE Trans Consum Electron 61(1):72–80

    Article  Google Scholar 

  23. Lee S. -W., Maik V, Jang J, Shin J, Paik J (2005) Noise-adaptive spatio-temporal filter for real-time noise removal in low light level images. IEEE Trans Consum Electron 51(2):648–653

    Article  Google Scholar 

  24. Li J, Li SZ, Pan Q, Yang T (2005) Illumination and motion-based video enhancement for night surveillance. In: 2nd Joint IEEE International workshop on visual surveillance and performance evaluation of tracking and surveillance, pp 169–175

  25. Li J, Yang T, Pan Q, Cheng Y (2009) Combining scene model and fusion for night video enhancement. J Electron (China) 26(1):88–93

    Article  Google Scholar 

  26. Li Y, Lu J, Wang J, Miao Z, Xu W (2013) Night vision image contrast enhancement base on adaptive dynamic histogram. In: 2013 Fourth international conference on digital manufacturing automation, pp 823–828

  27. Li Y, Tan RT, Brown MS (2015) Nighttime haze removal with glow and multiple light colors. In: Proceedings of the IEEE international conference on computer vision, pp 226–234

  28. Ling Z, Liang Y, Wang Y, Shen H, Lu X (2015) Adaptive extended piecewise histogram equalisation for dark image enhancement. IET Image Process 9 (11):1012–1019

    Article  Google Scholar 

  29. Łoza A, Bull DR, Hill PR, Achim AM (2013) Automatic contrast enhancement of low-light images based on local statistics of wavelet coefficients. Digital Signal Process 23(6):1856–1866

    Article  Google Scholar 

  30. Makwana I, Zaveri T, Gupta V (2011) Efficient color transfer method based on colormap clustering for night vision applications. In: 2011 Third national conference on computer vision, pattern recognition, image processing and graphics (NCVPRIPG). IEEE, pp 196–199

  31. Malm H, Oskarsson M, Warrant E, Clarberg P, Hasselgren J, Lejdfors C (2007) Adaptive enhancement and noise reduction in very low light-level video. In: IEEE 11th International conference on computer vision, 2007. ICCV 2007. IEEE, pp 1–8

  32. Meng Y, Kong D, Zhu Z, Zhao Y (2019) From night to day: Gans based low quality image enhancement, Neural Process Lett. [Online]. Available: https://doi.org/10.1007/s11063-018-09968-2

    Article  Google Scholar 

  33. Pan J, Sun D, Pfister H, Yang M-H (2016) Blind image deblurring using dark channel prior. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1628–1636

  34. Pei S-C, Lee T-Y (2012) Night time haze removal using color transfer pre-processing and dark channel prior. In: 2012 19th IEEE International conference on image processing. IEEE, pp 957–960

  35. Quevedo E, De La Cruz J, Callico GM, Tobajas F, Sarmiento R (2014) Video enhancement using spatial and temporal super-resolution from a multi-camera system. IEEE Trans Consum Electron 60(3):420–428

    Article  Google Scholar 

  36. Rao Y, Lin W, Chen L (2010) Image-based fusion for video enhancement of night-time surveillance. Optical Eng 49(12):120 501–120 501

    Article  Google Scholar 

  37. Rao Y, Chen Z, Sun M-T, Hsu Y-F, Zhang Z (2011) An effecive night video enhancement algorithm. In: 2011 IEEE visual communications and image processing (VCIP). IEEE, pp 1–4

  38. Rao Y, Hou L, Wang Z, Chen L (2014) Illumination-based nighttime video contrast enhancement using genetic algorithm. Multimed Tools Appl 70(3):2235–2254

    Article  Google Scholar 

  39. Rivera AR, Ryu B, Chae O (2012) Content-aware dark image enhancement through channel division. IEEE Trans Image Process 21(9):3967–3980

    Article  MathSciNet  Google Scholar 

  40. Soumya T, Thampi SM (2015) Day color transfer based night video enhancement for surveillance system. In: 2015 IEEE International conference on signal processing, informatics, communication and energy systems (SPICES). IEEE, pp 1–5

  41. Soumya T, Thampi SM (2016) Recolorizing dark regions to enhance night surveillance video. Multimed Tools Appl: 1–17

  42. Soumya T, Thampi SM (2017) A fuzzy fusion approach to enlighten the illuminated regions of night surveillance videos. J Intell Fuzzy Syst 32(4):3143–3149

    Article  Google Scholar 

  43. Soumya T, Thampi SM (2017) Self-organized night video enhancement for surveillance systems. SIViP 11(1):57–64

    Article  Google Scholar 

  44. Su H, Jung C, Wang L, Wang S, Du Y (2019) Adaptive tone mapping for display enhancement under ambient light using constrained optimization. Displays 56:11–22

    Article  Google Scholar 

  45. Wang Q, Ward RK (2007) Fast image/video contrast enhancement based on weighted thresholded histogram equalization. IEEE Trans Consum Electron 53(2)

    Article  Google Scholar 

  46. Wang W, Chen Z, Yuan X, Wu X (2019) Adaptive image enhancement method for correcting low-illumination images. Inform Sci

  47. Warrant E, Oskarsson M, Malm H (2014) The remarkable visual abilities of nocturnal insects: neural principles and bioinspired night-vision algorithms. Proc IEEE 102(10):1411–1426

    Article  Google Scholar 

  48. Xiao H, Rao Y (2015) Research on deep auto-encoder network for nighttime video enhancement. J Inf Comput Sci 12(10):4125–4136

    Article  Google Scholar 

  49. Xu Q, Jiang H, Scopigno R, Sbert M (2014) A novel approach for enhancing very dark image sequences. Signal Process 103:309–330

    Article  Google Scholar 

  50. Yamasaki A, Takauji H, Kaneko S, Kanade T, Ohki H (2008) Denighting: enhancement of nighttime images for a surveillance camera. In: 19th International conference on pattern recognition (ICPR), pp 1–4

  51. Yu J, Liao Q (2010) Color constancy-based visibility enhancement in low-light conditions. In: 2010 International conference on digital image computing: techniques and applications (DICTA). IEEE, pp 441–446

  52. Zhang X, Shen P, Luo L, Zhang L, Song J (2012) Enhancement and noise reduction of very low light level images. In: 2012 21st International conference on pattern recognition (ICPR). IEEE, pp 2034–2037

  53. Zhang J, Cao Y, Wang Z (2014) Nighttime haze removal based on a new imaging model. In: 2014 IEEE International conference on image processing (ICIP). IEEE, pp 4557–4561

  54. Zhang J, Cao Y, Wang Z (2016) Nighttime haze removal with illumination correction. arXiv:1606.01460

  55. Zhang Q, Nie Y, Zhang L, Xiao C (2016) Underexposed video enhancement via perception-driven progressive fusion. IEEE Trans Vis Comput Graph 22(6):1773–1785

    Article  Google Scholar 

  56. Zhang C, Shivakumara P, Xue M, Zhu L, Lu T, Pal U (2018) New fusion based enhancement for text detection in night video footage. In: Pacific rim conference on multimedia. Springer, pp 46–56

  57. Zhuo L, Hu X, Li J, Zhang J, Li X (2019) A naturalness-preserved low-light enhancement algorithm for intelligent analysis. Chin J Electron 28(2):316–324

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank University of Kerala, LBS Centre for Science and Technology, College of Engineering Trivandrum, College of Engineering Perumon and Centre for Engineering Research and Development for providing research facilities.

Author information

Authors and Affiliations

  1. University of Kerala, College of Engineering Perumon, Kerala, India

    Soumya T

  2. Indian Institute of Information Technology and Management, Kerala, India

    Sabu M. Thampi

Authors
  1. Soumya T
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sabu M. Thampi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Soumya T.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

T, S., Thampi, S.M. Nighttime visual refinement techniques for surveillance video: a review. Multimed Tools Appl 78, 32137–32158 (2019). https://doi.org/10.1007/s11042-019-07944-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07944-z

Keywords

Search

Navigation