Skip to main content
SpringerLink
Log in

Fast and Error-Bounded Space-Variant Bilateral Filtering

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

The traditional space-invariant isotropic kernel utilized by a bilateral filter (BF) frequently leads to blurry edges and gradient reversal artifacts due to the existence of a large amount of outliers in the local averaging window. However, the efficient and accurate estimation of space-variant kernels which adapt to image structures, and the fast realization of the corresponding space-variant bilateral filtering are challenging problems. To address these problems, we present a space-variant BF (SVBF), and its linear time and error-bounded acceleration method. First, we accurately estimate spacevariant anisotropic kernels that vary with image structures in linear time through structure tensor and minimum spanning tree. Second, we perform SVBF in linear time using two error-bounded approximation methods, namely, low-rank tensor approximation via higher-order singular value decomposition and exponential sum approximation. Therefore, the proposed SVBF can efficiently achieve good edge-preserving results. We validate the advantages of the proposed filter in applications including: image denoising, image enhancement, and image focus editing. Experimental results demonstrate that our fast and error-bounded SVBF is superior to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aurich V, Weule J. Non-linear Gaussian filters performing edge preserving diffusion. In Proc. the 1995 DAGMSymposium on Mustererkennung, September 1995, pp.538-545.

  2. Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In Proc. the 6th International Conference on Computer Vision, January 1998, pp.839-846.

  3. Zhang M, Gunturk B K. Multiresolution bilateral filtering for image denoising. IEEE Trans. Imge. Proc., 2008, 17(12): 2324-2333.

    Article  MathSciNet  MATH  Google Scholar 

  4. Zhang B Y, Allebach J P. Adaptive bilateral filter for sharpness enhancement and noise removal. IEEE Trans. Imge. Proc., 2008, 17(5): 664-678.

    Article  MathSciNet  Google Scholar 

  5. Durand F, Dorsey J. Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graphics, 2002, 21(3): 257-266.

    Article  Google Scholar 

  6. Yang Q X. Hardware-efficient bilateral filtering for stereo matching. IEEE Trans. Patt. Anal. Mach. Inte., 2014, 36(5): 1026-1032.

    Article  Google Scholar 

  7. He K M, Sun J, Tang X O. Guided image filtering. IEEE Trans. Patt. Anal. Mach. Inte., 2013, 35(6): 1397-1409.

    Article  Google Scholar 

  8. Choudhury P, Tumblin J. The trilateral filter for high contrast images and meshes. In Proc. the 2005 International Conference on Computer Graphics and Interactive Techniques, July 2005, Article No. 5.

  9. Porikli F. Constant time O(1) bilateral filtering. In Proc. the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2008, Article No. 505.

  10. Chaudhury K N. Acceleration of the shiftable O(1) algorithm for bilateral filtering and nonlocal means. IEEE Trans. Imge. Proc., 2013, 22(4): 1291-1300.

    Article  MathSciNet  MATH  Google Scholar 

  11. Dai L Q, Yuan M K, Zhang X P. Speeding up the bilateral filter: A joint acceleration way. IEEE Trans. Imge. Proc., 2016, 25(6): 2657-2672.

    Article  MathSciNet  MATH  Google Scholar 

  12. Popkin T, Cavallaro A, Hands D. Accurate and efficient method for smoothly space-variant Gaussian blurring. IEEE Trans. Imge. Proc., 2010, 19(5): 1362-1370.

    Article  MathSciNet  MATH  Google Scholar 

  13. Baek J, Jacobs D E. Accelerating spatially varying Gaussian filters. ACM Trans. Graphics, 2010, 29(6): Article No. 169.

  14. Muñoz-Barrutia A, Artaechevarria X, Ortiz-de-Solorzano C. Spatially variant convolution with scaled B-splines. IEEE Trans. Imge. Proc., 2010, 19(1): 11-24.

    Article  MathSciNet  MATH  Google Scholar 

  15. Chaudhury K N, Mu˜noz-Barrutia A, Unser M. Fast spacevariant elliptical filtering using box splines. IEEE Trans. Imge. Proc., 2010, 19(9): 2290-2306.

    Article  MATH  Google Scholar 

  16. Bao L C, Song Y B, Yang Q X, Yuan H, Wang G. Tree filtering: Efficient structure-preserving smoothing with a minimum spanning tree. IEEE Trans. Imge. Proc., 2014, 23(2): 555-569.

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhang S, Sheng H, Li C, Zhang J, Xiong Z. Robust depth estimation for light field via spinning parallelogram operator. Computer Vision and Image Understanding, 2016, 145: 148-159.

    Article  Google Scholar 

  18. Sheng H, Zhang S, Cao X C, Fang Y J, Xiong Z. Geometric occlusion analysis in depth estimation using integral guided filter for light-field image. IEEE Trans. Imge. Proc., 2017, 26(12): 5758-5771.

    Article  MathSciNet  MATH  Google Scholar 

  19. Shapiro L G, Stockman G C. Computer Vision: Theory and Applications, 2001.

  20. Liu Q, Zhang C M, Guo Q, Zhou Y F. A nonlocal gradient concentration method for image smoothing. Computational Visual Media, 2015, 1(3): 197-209.

    Article  Google Scholar 

  21. Ma G H, Zhang M L, Li X M, Zhang C M. Image smoothing based on image decomposition and sparse high frequency gradient. Journal of Computer Science and Technology, 2018, 33(3): 502-510.

    Article  MathSciNet  Google Scholar 

  22. Lu J B, Shi K Y, Min D B, Lin L, Do M N. Cross-based local multipoint filtering. In Proc. the 2012 IEEE Conference on Computer Vision and Pattern Recognition, June 2012, pp.430-437.

  23. Tan X, Sun C M, Pham T D. Multipoint filtering with local polynomial approximation and range guidance. In Proc. the 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp.2941-2948.

  24. Dai L Q, Yuan M K, Zhang F H, Zhang X P. Fully connected guided image filtering. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.352-360.

  25. Gunturk B K. Fast bilateral filter with arbitrary range and domain kernels. IEEE Trans. Imge. Proc., 2011, 20(9): 2690-2696.

    Article  MathSciNet  MATH  Google Scholar 

  26. Yuan M K, Zhang X P. Bilateral filter acceleration based on weighted variable projection. Electronics Letters, 2018, 54(6): 352-353.

    Article  Google Scholar 

  27. Getreuer P. A survey of Gaussian convolution algorithms. Image Processing on Line, 2013, 3: 286-310.

    Article  Google Scholar 

  28. Tan S, Dale J L, Johnston A. Performance of three recursive algorithms for fast space-variant Gaussian filtering. Real-Time Imaging, 2003, 9(3): 215-228.

    Article  Google Scholar 

  29. Xu L, Ren J, Yan Q, Liao R J, Jia J Y. Deep edge-aware filters. In Proc. the 32nd International Conference on Machine Learning, July 2015, pp.1669-1678.

  30. Chen J W, Adams A, Wadhwa N, Hasinoff S W. Bilateral guided upsampling. ACM Trans. Graphics, 2016, 35(6): Article No. 203.

  31. Liu S F, Pan J S, Yang M H. Learning recursive filters for low-level vision via a hybrid neural network. In Proc. the 14th European Conference on Computer Vision, Part IV, October 2016, pp.560-576.

  32. Chen Q F, Xu J, Koltun V. Fast image processing with fully-convolutional networks. In Proc. the 2017 IEEE International Conference on Computer Vision, October 2017, pp.2516-2525.

  33. Wu H K, Zheng S, Zhang J G, Huang K Q. Fast end-to-end trainable guided filter. In Proc. the 2018 IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.1838-1847.

  34. Xie J Y, Xu L L, Chen E H. Image denoising and inpainting with deep neural networks. In Proc. the 26th Annual Conference on Neural Information Processing Systems, December 2012, pp.350-358.

  35. Meinhardt T, M¨oller M, Hazirbas C, Cremers D. Learning proximal operators: Using denoising networks for regularizing inverse imaging problems. In Proc. the 2017 IEEE International Conference on Computer Vision, October 2017, pp.1799-1808.

  36. Zhang K, Zuo W M, Chen Y J, Meng D Y, Zhang L. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans. Imge. Proc., 2017, 26(7): 3142-3155.

    Article  MathSciNet  MATH  Google Scholar 

  37. Zhang K, Zuo W M, Zhang L. FFDNet: Toward a fast and flexible solution for CNN based image denoising. IEEE Trans. Imge. Proc., 2018, 27(9): 4608-4622.

    Article  MathSciNet  MATH  Google Scholar 

  38. Lehtinen J, Munkberg J, Hasselgren J, Laine S, Karras T, Aittala M, Aila T. Noise2noise: Learning image restoration without clean data. arXiv:1803.04189, 2018. https://arxiv.org/abs/1803.04189, March 2019.

  39. Xu L, Ren J S J, Liu C, Jia J. Deep convolutional neural network for image deconvolution. In Proc. the 2014 Annual Conference on Neural Information Processing Systems, December 2014, pp.1790-1798.

    Google Scholar 

  40. Pan J S, RenWQ, Hu Z, Yang M H. Learning to deblur images with exemplars. IEEE Trans. Patt. Anal. Mach. Inte. doi: https://doi.org/10.1109/TPAMI.2018.2832125.

  41. Shen Z Y, Lai W S, Xu T F, Kautz J, Yang M H. Deep semantic face deblurring. arXiv:1803.03345, 2018. https://arxiv.org/abs/1803.03345, March 2019.

  42. Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.1646-1654.

  43. Dong C, Loy C C, He K M, Tang X O. Image superresolution using deep convolutional networks. IEEE Trans. Patt. Anal. Mach. Inte., 2016, 38(2): 295-307.

    Article  Google Scholar 

  44. Lai W S, Huang J B, Ahuja N, Yang M H. Fast and accurate image super-resolution with deep Laplacian pyramid networks. IEEE Trans. Patt. Anal. Mach. Inte. doi: https://doi.org/10.1109/TPAMI.2018.2865304.

  45. Lucas A, Iliadis M, Molina R, Katsaggelos A K. Using deep neural networks for inverse problems in imaging: Beyond analytical methods. IEEE Signal Processing Magazine, 2018, 35(1): 20-36.

    Article  Google Scholar 

  46. Wang Z, Bovik A C. Foveated image and video coding. In Digital Video, Image Quality and Perceptual Coding (1st edition), Wu H R, Rao K R (eds.), CRC Press, 2005, pp.431-457.

  47. Chaudhury K N. Optimally localized wavelets and smoothing kernels [Ph.D. Thesis]. Swiss Federal Institute of Technology Lausanne, 2011.

  48. Bernad J. Digital Image Processing. Springer, 1997.

  49. Deledalle C A, Denis L, Tabti S, Tupin F. Closed-form expressions of the eigen decomposition of 2 × 2 and 3 × 3 Hermitian matrices. Technical Report, Université de Lyon, 2017. https://hal.archives-ouvertes.fr/hal-01501-221/file/matrix exp and log formula.pdf, March 2019.

  50. de Lathauwer L, de Moor B, Vandewalle J. A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 2000, 21(4): 1253-1278.

    Article  MathSciNet  MATH  Google Scholar 

  51. Golub G, Pereyra V. Separable nonlinear least squares: The variable projection method and its applications. Inverse Problems, 2003, 19(2): Article No. R1.

  52. Beylkin G, Monzón L. On approximation of functions by exponential sums. Applied and Computational Harmonic Analysis, 2005, 19(1): 17-48.

    Article  MathSciNet  MATH  Google Scholar 

  53. Deriche R. Recursively implementating the Gaussian and its derivatives. Technical Report, Institut National de Recherche en Informatique et en Automatique, 1993. https://hal.inria.fr/file/index/docid/74778/filename/RR-1893.pdf, March 2019.

  54. Young I T, van Vliet L J. Recursive implementation of the Gaussian filter. Sign. Proc., 1995, 44(2): 139-151.

    Article  Google Scholar 

  55. Young I T, van Uliet L J, van Ginkel M. Recursive Gabor filtering. IEEE Trans. Sign. Proc., 2002, 50(11): 2798-2805.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Meng-Ke Yuan, Dong-Ming Yan & Xiao-Peng Zhang

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Meng-Ke Yuan, Dong-Ming Yan, Jun Xiao & Xiao-Peng Zhang

  3. School of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing, 210094, China

    Long-Quan Dai

  4. State Key Laboratory of Remote Sensing Science, School of Geography, Beijing Normal University, Beijing, 100875, China

    Li-Qiang Zhang

Authors
  1. Meng-Ke Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Long-Quan Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong-Ming Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Li-Qiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiao-Peng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao-Peng Zhang.

Electronic supplementary material

ESM 1

(PDF 741 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, MK., Dai, LQ., Yan, DM. et al. Fast and Error-Bounded Space-Variant Bilateral Filtering. J. Comput. Sci. Technol. 34, 550–568 (2019). https://doi.org/10.1007/s11390-019-1926-8

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-019-1926-8

Keywords

Search

Navigation