Skip to main content
SpringerLink
Log in

Real-time video denoising on multicores and GPUs with Kalman-based and Bilateral filters fusion

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In the context of video processing, image noise caused by acquisition, transfer and image compression can be attenuated by video denoising algorithms. However, their computational cost must be as low as possible to allow them to be applied to real-time applications. In this paper, we propose stmkf, a real-time video denoising algorithm based on Kalman and Bilateral filters. We evaluate the effectiveness of stmkf using several common videos used in the literature and we compare it to other denoising algorithms using both the PSNR and SSIM metrics. Our experimental results show that stmkf is competitive with other filters, especially for videos that feature stationary backgrounds such as in videoconferencing, video lectures and video surveillance. We also evaluate the performance of our parallel implementations of stmkf for CPUs and GPUs. stmkf achieved a performance improvement of up to \(2.9\times \) on a Intel i7 multicore processor with 4 cores compared to the sequential solution. The results obtained with the GPU version of stmkf on a NVIDIA Tesla K40 showed a performance improvement of up to \(7.6\times \) compared to the Intel i7 multicore processor.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Source codes available at: http://github.com/sergiogenilson/STMKF.

  2. The GPU processing power is usually measured by the number of floating-point operations it can issue per second (FLOPS).

  3. Source code available at http://teacher.buet.ac.bd/mahbubur/resources/TCSVT_prog.rar.

References

  1. Bardu, T.: Variational image denoising approach with diffusion porous media flow. Abstr. Appl. Anal. 2013, 8 (2013)

    MathSciNet  Google Scholar 

  2. Buades, A., Coll, B., Morel, J.-M.: Nonlocal image and movie denoising. Int. J. Comput. Vision 76(2), 123–139 (2008)

    Article  Google Scholar 

  3. Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: Conference on Computer Vision and Pattern Recognition, CVPR ’05, pp. 60–65. Washington, DC, USA, IEEE Computer Society (2005)

  4. Chan, T.-W., Au, O.C., Chong, T.-S., Chau, W.-S.: A novel content-adaptive video denoising filter. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 649–652, Philadelphia, USA (2005)

  5. Chaudhury, K.N.: Acceleration of the shiftable algorithm for bilateral filtering and nonlocal means. IEEE Trans. Image Process. 22(4), 1291–1300 (2013)

    Article  MathSciNet  Google Scholar 

  6. Chen, T.-Y., Chen, T.-H., Su, C.-P., Chen, Y.-J.: The study on video enhancement in the low-light environment by spatio-temporal filtering. In: International Conference on Intelligent Systems Design and Applications (ISDA), vol. 3, pp. 561–564, Kaohsiung, Taiwan (2008)

  7. Chenglin Z., Yu, L., Xin, T., Wei, W., Maojun, Z. (2013) Video denoising based on a spatiotemporal Kalman-bilateral mixture model. Sci. World J. 2013 (2013)

  8. Dabov, K., Foi, A., Egiazarian, K.: Video denoising by sparse 3D transform-domain collaborative filtering. In: European Signal Processing Conference, pp. 145–149, Poznan, Poland. IEEE (2007)

  9. Davis, L., Rosenfeld, A.: Noise cleaning by iterated cleaning. IEEE Trans. Syst. Man Cybern. SMC 8(9), 705–710 (1978)

    Article  Google Scholar 

  10. Dufaux, F., Callet, P.L., Mantiuk, R., Mrak, M.:  High Dynamic Range Video: From Acquisition, to Display and Applications. Elsevier (2016). ISBN 9780128030394

  11. Farooque, M.A., Sohankar, J.S.: Survey on various noises and techniques for denoising the color image. Int. J. Appl. Innov. Eng. Manage. (IJAIEM) 2, 217 (2013)

    Google Scholar 

  12. Garg, R., Kumar, A.: Comparision of various noise removals using bayesian framework. Int. J. Mod. Eng. Res. (IJMER) 2, 265 (2012)

    Google Scholar 

  13. Han, Y., Chen, R.: Efficient video denoising based on dynamic nonlocal means. Image Vision Comput. 30, 78–85 (2012)

    Article  Google Scholar 

  14. Hong-Zhi, W., Ling, C., Shu-Liang, X.: Improved video denoising algorithm based on spatial-temporal combination. In: International Conference on Image and Graphics (ICIG), pp. 64–67, Qingdao, China. IEEE (2013)

  15. Jojy, C., Nair, M.S., Subrahmanyam, G.R.K.S., Raji, R.: Discontinuity adaptive non-local means with importance sampling unscented Kalman filter for de-speckling SAR images. IEEE J. Sel. Top. Appl. Earth Obser. Remote Sens. 6(4), 1964–1970 (2013)

    Article  Google Scholar 

  16. Jung, B., Sukhatme, G.S.: Detecting moving objects using a single camera on a mobile robot in an outdoor environment. In: International Conference on Intelligent Autonomous Systems, pp. 980–987 (2004)

  17. Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82(D), 35–45 (1960)

    Article  MathSciNet  Google Scholar 

  18. Karnati, V., Uliyar, M., Dey, S.: Fast non-local algorithm for image denoising. In: International Conference on Image Processing (ICIP), pp. 3873–3876. IEEE (Nov 2009)

  19. Kirk, D.B., Wen-mei W.H.: Programming Massively Parallel Processors: A Hands-on Approach. Morgan Kaufmann Publishers Inc., San Francisco, 1st edn. (2010). ISBN 0123814723

  20. Kokkonis, G., Psannis, K.E., Roumeliotis, M., Ishibashi, Y.: Efficient Algorithm for transferring a real-time HEVC stream with haptic data through the internet. J. Real-Time Image Process. pp. 1–13, (2015). ISSN 1861-8219. doi:10.1007/s11554-015-0505-7

    Article  Google Scholar 

  21. Kokkonis, G., Psannis, K.E., Roumeliotis, M., Schonfeld, D.: Real-time wireless multisensory smart surveillance with 3D-HEVC streams for internet-of-things (iot). J. Supercomput. pp 1–19, (2016). ISSN 1573-0484. doi:10.1007/s11227-016-1769-9

    Article  Google Scholar 

  22. Kostadin D., Alessandro F., Vladimir K., Karen E.: Image denoising with block-matching and 3D filtering. In: SPIE-IS&T Electronic Imaging, p. 6064 (2006)

  23. Li, W., Zhang, J., Dai, Q.: Video denoising using shape-adaptive sparse representation over similar spatio-temporal patches. Signal Proc.: Image. Communication 26(4–5), 250–265 (2011)

    Google Scholar 

  24. Li, X., Zheng, Y.: Patch-based video proc.: a variational bayesian approach. IEEE Trans. Circuits Syst Video Technol 19(1), 27–40 (2009)

    Article  Google Scholar 

  25. Mahmoud, R.O., Faheem, M.T., Sarhan, A.: Intelligent denoising technique for spatial video denoising for real-time applications. In: International Conference on Computer Engineering Systems (ICCES), pp. 407–412, Cairo, Egypt. IEEE (2008)

  26. Mahmoudi, M., Sapiro, G.: Fast image and video denoising via nonlocal means of similar neighborhoods. IEEE Signal Process. Lett. 12(12), 839–842 (2005)

    Article  Google Scholar 

  27. Memos, V.A., Psannis, K.E.: Encryption algorithm for efficient transmission of hevc media. J. Real-Time Image Process. pp. 1–10, (2015). ISSN 1861-8219. doi:10.1007/s11554-015-0509-3

    Article  Google Scholar 

  28. Mitchell, H.B., Mashkit, N.: Noise smoothing by a fast k-nearest neighbour algorithm. Signal Process. Image Commun. 4(3), 227–232 (1992)

    Article  Google Scholar 

  29. OpenMP Architecture Review Board. OpenMP application program interface version 4.0, July 2013. URL http://www.openmp.org/mp-documents/OpenMP4.0.0.pdf

  30. Pauwels, K., Tomasi, M., Alonso, J. Diaz., Ros, E., Van Hulle, M. M.: A comparison of fpga and GPU for real-time phase-based optical flow, stereo, and local image features. IEEE Trans. Comput. 61(7): 999–1012, (2012). ISSN 0018-9340

    Article  MathSciNet  Google Scholar 

  31. Pizurica, A., Zlokolica, V., Philips, W.: Noise reduction in video sequences using wavelet-domain and temporal filtering. In: Photonics Technologies for Robotics, Automation, and Manufacturing, Int. Soc. for Optics and Photonics, pp. 48–59 (2004)

  32. Psannis, K.E.: Hevc in wireless environments. J. Real-Time Image Process. pp. 1–8, (2015). ISSN 1861-8219. doi:10.1007/s11554-015-0514-6

    Article  Google Scholar 

  33. Pulli, K., Baksheev, A., Kornyakov, K., Eruhimov, V.: Real-time computer vision with OpenCV. Commun. ACM 55(6): 61–69, (2012). ISSN 0001-0782

    Article  Google Scholar 

  34. Rahman, S.M.M., Ahmad, M.O., Swamy, M.N.S.: Video denoising based on inter-frame statistical modeling of wavelet coefficients. IEEE Trans. Circuits Syst. Video Technol. 17(2), 187–198 (2007)

    Article  Google Scholar 

  35. Ryu, J., Nishimura, T. H.: Fast image blurring using lookup table for real time feature extraction. In: 2009 IEEE International Symposium on Industrial Electronics, pp. 1864–1869 (2009)

  36. Seiller, N., Singhal, N., Park, I.K.: Object oriented framework for real-time image processing on GPU. In: International Conference on Image Processing (ICIP), pp. 4477–4480, Hong Kong, China. IEEE (2010)

  37. Selesnick, I.W, Li, K.Y.: Video denoising using 2D and 3D dual-tree complex wavelet transforms. In: Annual Meeting on Optical Science and Technology (SPIE), Int. Soc. for Optics and Photonics, pp. 607–618. (2003)

  38. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: International Conference on Computer Vision, Bombay, India, pp. 839–846, IEEE (1998)

  39. Van De Ville, D., Kocher, M.: SURE-based non-local means. IEEE Signal Process. Lett. 16(11), 973–976 (2009)

    Article  Google Scholar 

  40. Wang, Z., Bovik, A.C., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Proc. 13(4), 600 (2004)

    Article  Google Scholar 

  41. Wolf, W., Ozer, B., Lv, T.: Smart cameras as embedded systems. Computer 35(9), 48–53 (2002)

    Article  Google Scholar 

  42. Zlokolica, V., Pizurica, A., Philips, W.: Wavelet-domain video denoising based on reliability measures. IEEE Trans. Circuits Syst. Video Technol. 16(8), 993–1007 (2006)

    Article  Google Scholar 

  43. Zlokolica, V., Philips, W., Van De Ville, D.: A new non-linear filter for video processing. In: IEEE Benelux Signal Processing Symposium, pp. 221–224 (2002)

Download references

Author information

Authors and Affiliations

  1. Department of Informatics and Statistics (INE), Federal University of Santa Catarina (UFSC), Campus Universitário Reitor João David Ferreira Lima, Trindade, Cx.P. 476, Florianópolis, 88040-900, Brazil

    Sergio G. Pfleger, Patricia D. M. Plentz, Alyson D. Pereira & Márcio Castro

  2. Computer Science Department, Pontifical Catholic University of Minas Gerais (PUC Minas), Avenida Dom José Gaspar, 500, Belo Horizonte, 30535-610, Brazil

    Rodrigo C. O. Rocha

Authors
  1. Sergio G. Pfleger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patricia D. M. Plentz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rodrigo C. O. Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alyson D. Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Márcio Castro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Patricia D. M. Plentz.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 129997 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pfleger, S.G., Plentz, P.D.M., Rocha, R.C.O. et al. Real-time video denoising on multicores and GPUs with Kalman-based and Bilateral filters fusion. J Real-Time Image Proc 16, 1629–1642 (2019). https://doi.org/10.1007/s11554-016-0659-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-016-0659-y

Keywords

Search

Navigation