Abstract
With the advent of embedded vision systems, smart sensors with integrated image signal processing (ISP) become a hot topic. This poses a need for efficient hardware implementation, regarding resource utilization and power consumption, of core image processing algorithms. Power consumption is especially important, since many of the target devices are usually battery operated. Edge-aware filtering, although it is used in many core image processing algorithms, is still challenging operation, especially in cases where large kernels are needed. In this paper, efficient hardware realization of fast guided filter (FGF) is proposed. It is based on idea that large filter of size \(R=K \cdot S\) can be calculated by downsampling input image by factor S and using filter of size K. Besides reduced memory and logic requirements, this optimization enables that, for the scaling factor S, core processing is done at \(1/S^{2}\) pixel clock, providing significantly lower power consumption. Experimental results on Cyclone V FPGA chip demonstrate that, for FGF of size \(35 \times 35\) with downsampling factor \(S=7\), the proposed design achieves 60 fps for 1080p video. Memory utilization is 147.3 kB without need for any off-chip memory. Core dynamic power consumption is 79.89 mW. Proposed design consumes less total power than state-of-the-art guided filter realizations including ASIC-based solutions. This module can be seamlessly integrated into smart sensors ISP units, because it is designed for power-efficient streaming processing.




















Similar content being viewed by others
References
Barry, B., Brick, C., Connor, F., Donohoe, D., Maloney, D., Richmond, R., ORiordan, M., Toma, V.: Always-on vision processing unit for mobile applications. IEEE Micro 35, 56–66 (2015)
Charoensak, C., Sattar, F.: FPGA design of a real-time implementation of dynamic range compression for improving television picture. In: Proc. 6th IEEE ICICSP, pp. 1–5 (2007)
Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph 21, 257–266 (2002)
Eilersten, G., Mantiuk, R.K., Unger, J.: A comparative review of tone-mapping algorithms for high dynamic range video. Comput. Gr. Forum 36, 565–592 (2017)
El Mezeni, D., Saranovac, L.: Fast self-guided filter with decimated box filters. In: Proc. INFOTEH-JAHORINA, pp. 633–638 (2016)
El Mezeni, D., Saranovac, L.: Enhanced local tone mapping for detail preserving reproduction of high dynamic range images. J. Vis. Comm. Imag. Rep 53, 122–133 (2018)
Ercegovac, M., Lang, T.: Digital Arithmetic. Morgan Kaufmann, Francisco (2004)
Gabiger-Rose, A., Kube, M., Weigel, R., Rose, R.: An FPGA-based fully synchronized design of a bilateral filter for real-time image denoising. IEEE Trans. Ind. Electron. 64, 4093–4104 (2014)
Hartmann, C., Fey, D.: An extended analysis of memory hierarchies for efficient implementations of image processing applications. J. Real Time Image Process. 14, 713–728 (2018)
He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1397–1409 (2013)
He, K., Sun, J.: Fast guided filter. arXiv preprint arXiv:1505.00996 (2015)
Hristova, H., Meur, O., Cozot, R., Bouatouch, K.: High-dynamic-range image recovery from flash and non-flash image pairs. Vis. Comput. Int. J. Comp. Graph 33, 725–735 (2017)
Igarashi, M., Ikebe, M., Shimoyama, S., Yamano, K., Motohisa, J.: O(1) bilateral filtering with low memory usage. In: Proc. IEEE 17th ICIP, pp. 3301–3304 (2010)
Johnson, B., Moncy, J.K., Rani, J.S.: Self adaptable high throughput reconfigurable bilateral filter architectures for real-time image de-noising. J. Real Time Image Process. 1–20 (2017)
Kao, C.C., Lai, J.H., Chien, S.Y.: VLSI architecture design of guided filter for 30 frames/s full-HD video. IEEE Trans. Circ. Syst. Video Tech. 24, 513–524 (2014)
Lapray, P., Heyrman, B., Ghinhac, D.: HDR-ARtiSt: an adaptive real-time smart camera for high dynamic range imaging. J. Real Time Image Process. 12, 747–762 (2016)
Li, S., Kang, X., Hu, J.: Image fusion with guided filtering. IEEE Trans. Image Process. 22, 2864–2875 (2013)
Marsi, S., Saponara, S.: Integrated video motion estimator with Retinex-like pre-processing for robust motion analysis in automotive scenarios: algorithmic and real-time architecture design. J. Real Time Image Process. 5, 275–289 (2010)
Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. Int. J. Comput. Vis. 81, 24–52 (2009)
Popovic, D., Seyid, K., Pignat, E., Cogal, O., Leblebici, Y.: Multi-camera platform for panoramic real-time HDR video construction and rendering. J. Real Time Image Process. 12, 697–708 (2016)
Porikli, F.: Constant time O(1) bilateral filtering. In: Proc. IEEE CVPR, pp. 1–8 (2008)
Saponara, S., Fanucci, L., Marsi, S., Ramponi, G.: Algorithmic and architectural design for real-time and power-efficient Retinex image/video processing. J. Real Time Image Process. 1, 267–283 (2007)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proc. IEEE 6th ICCV, pp. 839–846 (1998)
Tseng, Y.C., Hsu, P.H., Chang, T.S.: A 124 Mpixels/s VLSI design for histogram-based joint bilateral filtering. IEEE Trans. Image Process. 20, 3231–3241 (2011)
Ttofis, C., Theocharides, T.: High-quality real-time hardware stereo matching based on guided image filtering. In: Proc. IEEE DATE, p. 356 (2014)
Vinh, T.Q., Park, J.H., Kim, Y.C., Hong, S.H.: FPGA implementation of real-time edge-preserving filter for video noise reduction. In: Proc. IEEE ICCEE, pp. 611–614 (2008)
Weiss, B.: Fast median and bilateral filtering. ACM Trans. Graph 25, 519–526 (2006)
Wu, L., Jong, C.C.: A VLSI architecture for real-time gradient guided image filtering. In: Proc. IFIP/IEEE VLSI-SoC, pp. 1–6 (2016)
Yang, Q., Tan, K.H., Ahuja, N.: Real-time O(1) bilateral filtering. In: Proc. IEEE CVPR, pp. 557–564 (2009)
Yang, Q., Ahuja, N., Tan, K.H.: Constant time median and bilateral filtering. Int. J. Comput. Vis. 112, 307 (2015)
Yang, Q.: Hardware-efficient bilateral filtering for stereo matching. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1026–1032 (2015)
Zhang, X., Sun, H., Chen, S., Zheng, N.: VLSI architecture exploration of guided image filtering for 1080p@60Hz video processing. IEEE Trans. Circ. Syst. Video Tech. 1, 230–241 (2018)
Zhu, S., Gao, R., Li, Z.: Stereo matching algorithm with guided filter and modified dynamic programming. Multim. Tools App. 76, 199–216 (2017)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
El Mezeni, D., Saranovac, L. Fast guided filter for power-efficient real-time 1080p streaming video processing. J Real-Time Image Proc 17, 511–525 (2020). https://doi.org/10.1007/s11554-018-0802-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-018-0802-z