Abstract
The deblocking filter in the standard H.265 SHVC reduces blocking artifacts at the edges and block boundaries of coded frame with exhaustive computation. But the maximum removal of these blocking artifacts is not achieved; also the computational complexity is still a burden. This made many research works discussed related to maximum removal of blocking artifacts without considering computational complexity. In this paper, a convolutional neural network (CNN)-based deblocking filter is proposed for SHVC in H.265 that removes blocking artifacts with less computation. The proposed CNN framework learns the reconstructed samples of the input frame in a video sequence. The blocking artifacts are efficiently detected by the preprocessing unit which is considered as the first convolution layer. Next, the features are extracted for the frames in a video based on patch by employing kernel and stride for scanning the complete frames in a video. The normalization is applied from base layer to enhancement layer in the CNN framework to preserve sharpness in the video by removing artifacts generated due to inter-layer prediction and quantization. The network model is trained by rectified linear unit activation function to perform nonlinear mappings. In addition, the CNN-based deblocking filter achieves less computation due to max-pooling and soft-max layers in CNN. The simulation results are conducted that produce an average of 0.76 dB increase in PSNR and 57% time saving compared with the standard SHM reference encoder.
Similar content being viewed by others
References
Park, C.-S., Kim, B.-G.: Performance analysis of inter-layer prediction in scalable extension of HEVC (SHVC) for adaptive media service. Displays 44, 27–36 (2016)
Pourazad, M.T., Doutre, C., Azimi, M., Nasiopoulos, P.: HEVC: the new gold standard for video compression. IEEE Consum. Electron. Mag. 1(3), 36–46 (2012)
Test model for scalable extensions of high efficiency video coding (HEVC), ISO/IEC and JTC1/SC29/WG11 (2013)
Larochelle, H., Bengio, Y., Louradour, J., Lamblin, P.: Exploring strategies for training deep neural networks. J. Mach. Learn. Res. 10, 1–40 (2009)
Lee, H., Ekanadham, C., Ng, A.Y.: Sparse deep belief net model for visual area v2. In: Proceedings of the 20th International Conference on Neural Information Processing Systems, pp. 873–880 (2007)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Swietojanski, P., Ghoshal, A., Renals, S.: Convolutional neural networks for distant speech recognition. IEEE Signal Process. Lett. 21(9), 1120–1124 (2014)
Wang, Z., Yan, D., Wang, R., Xiang, L., Wu, T.: Speech resampling detection based on inconsistency of band energy. Comput. Mater. Contin. 56(2), 247–259 (2018)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, ser. NIPS’12, 2012, vol. 1, pp. 1097–1105
Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1256–1272 (2017)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)
Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: a persistent memory network for image restoration. In: IEEE International Conference on Computer Vision (ICCV), pp. 4549–4557 (2018)
Lu, X., Martin, G.R.: Fast mode decision algorithm for the H.264/AVC scalable video coding extension. IEEE Trans. Circuits Syst. Video Technol. 23(5), 846–855 (2013)
Balaji, L., Thyagharajan, K.K.: H.264/SVC mode decision based on mode correlation and desired mode list. Int. J. Autom. Comput. 11(5), 510–516 (2014)
Khairy, M., Hamdy, A., Elsayed, A., Farouk, H.: Efficient enhancement for spatial scalable video coding transmission. Math. Probl. Eng. 2017, 1–13 (2017)
Balaji, L., Thyagharajan, K.K.: H.264 SVC complexity reduction based on likelihood mode decision. Sci. World J. 2015, 1–11 (2015)
Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: Content adaptive complexity reduction scheme for quality/fidelity scalable HEVC. In: Proceedings of ICASSP, pp. 1744–1748 (2013)
Content adaptive complexity reduction scheme for quality/fidelity scalable HEVC, ISO/IEC JTC1/SC29/WG11, L0042 (2013)
Tohidypour, H.R., Pourazad, M.T., Bashashati, H., Nasiopoulos, P.: Fast mode assignment for quality scalable extension of the high efficiency video coding (HEVC) standard: a Bayesian approach. In: Proceedings of BCI, pp. 61–65 (2013)
Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: Adaptive search range method for spatial scalable HEVC. In: Proceedings of IEEE International Conference on Consumer Electronics, pp. 191–192 (2014)
Bailleul, R., De Cock, J., Van de Walle, R.: Fast mode decision for SNR scalability in SHVC digest of technical papers. In: Proceedings of IEEE International Conference on Consumer Electronics, pp. 193–194 (2014)
Cote, G., Erol, B., Gallant, M., Kossentini, F.: H.263 + : video coding at low bit rates. IEEE Trans. Circuits Syst. Video Technol. 8(7), 849–866 (1998)
Sze, V., Budagavi, M., Madhukar, S., Sullivan, G.: High efficiency video coding (HEVC): algorithms and architectures. In: Chandrakasan, A.P. (ed.) Integrated Circuits and Systems. Springer, New York (2014)
Wien, M.: High efficiency video coding: coding tools and specification. In: Celebi, E., Chen, J., Gopi, E.S., Neustein, A., Poor, H.V. (eds.) Signals and Communication Technology. Springer, NewYork (2014)
Sullivan, G.J., Ohm, J., Woo-Jin, J.H., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)
Aminlou, A., Lainema, J., Ugur, K., Hannuksela, M.M., Gabbouj, M.: Differential coding using enhanced inter-layer reference picture for the scalable extension of H.265/HEVC video codec. IEEE Trans. Circuits Syst. Video Technol. 24(11), 1945–1956 (2014)
Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: Probabilistic approach for predicting the size of coding units in the quad-tree structure of the quality and spatial scalable HEVC. IEEE Trans. Multimed. 18(2), 182–195 (2016)
Shen, L., An, P., Feng, G.: Low-complexity scalable extension of the high-efficiency video coding (SHVC) encoding system. ACM Trans. Multimed. Comput. Commun. Appl. 15(2), 44 (2019)
Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: An encoder complexity reduction scheme for quality/fidelity scalable HEVC. IEEE Trans. Broadcast. 62(3), 664–674 (2016)
Raja, G., Mirza, M.J.: In-loop deblocking filter for H.264/AVC video. In: 2nd International Symposium on Communications, Control, and Signal Processing, ISCCSP (2006)
Norkin, A., Bjøntegaard, G., Fuldseth, A., Narroschke, M., Ikeda, M., Andersson, K., Zhou, M., Van der Auwera, G.: HEVC deblocking filter. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1746–1754 (2012)
Wang, Y., Guo, X., Fan, X.: Parallel in-loop filtering in HEVC encoder on GPU. IEEE Trans. Consum. Electron. 64(3), 276–284 (2018)
Lee, Y., Kim, J., Choi, Y., Kim, B.: CNN-based approach for visual quality improvement on heVC. In: IEEE International Conference on Consumer Electronics (ICCE 2018), pp. 498–500 (2018)
Balaji, L., Thyagharajan, K.K.: An enhanced performance for H.265/SHVC based on combined AEGBM3D filter and back-propagation neural network. Signal Image Video Process. 12(5), 809–817 (2018)
Sheen, L., Feng, G.F., An, P.A.: SHVC CU processing aided by a feed forward neural network. IEEE Trans. Ind. Inform. 99, 1 (2019)
LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)
List, P., Joch, A., Lainema, J., Bjontegaard, G., Karczewicz, M.: Adaptive deblocking filter. IEEE Trans. Circuits Syst. Video Technol. 13(7), 614–619 (2003)
Reeve III, H.C., Lim, J.S.: Reduction of blocking effects in image coding. Opt. Eng. 23(1), 230134 (1984)
Wang, C., Zhou, J., Liu, S.: Adaptive non-local means filter for image deblocking. Signal Process. Image Commun. 28(5), 522–530 (2013)
Liew, A.-C., Yan, H.: Blocking artifacts suppression in block-coded images using overcomplete wavelet representation. IEEE Trans. Circuits Syst. Video Technol. 14(4), 450–461 (2004)
Foi, A., Katkovnik, V., Egiazarian, K.: Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images. IEEE Trans. Image Process. 16(5), 1395–1411 (2007)
Jancsary, J., Nowozin, S., Rother, C.: Loss-specific training of nonparametric image restoration models: a new state of the art. In: European Conference on Computer Vision, pp. 112–125 (2012)
Li, Y., Guo, F., Tan, R.T., Brown, M.S.: A contrast enhancement framework with jpeg artifacts suppression. In: European Conference on Computer Vision, pp. 174–188 (2014)
SHM-7.0. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svnSHVCSoftware/tags/SHM-7.0, Accessed 30 Oct 2015
Bjontegaard, G.: Calculation of average PSNR differences between RDcurves. In: ITU-T Q. 6/SG16 VCEG, 15th Meeting, Austin, Texas, USA (2001)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Dhanalakshmi, A., Nagarajan, G. Convolutional Neural Network-based deblocking filter for SHVC in H.265. SIViP 14, 1635–1645 (2020). https://doi.org/10.1007/s11760-020-01713-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-020-01713-4