Abstract
Recently deep learning techniques have shown remarkable progress in image/video super-resolution. These techniques can be employed in a video coding system for improving the quality of the decoded frames. However, different from the conventional super-resolution works, the compression artifacts in the decoded frames should be concerned with. The straightforward solution is to integrate the artifacts removing techniques before super-resolution. Nevertheless, some helpful features may be removed together with the artifacts, and remaining artifacts can be exaggerated. To address these problems, we design an end-to-end restoration-reconstruction deep neural network (RR-DnCNN) using the degradation-aware techniques. RR-DnCNN is applied to the down-sampling based video coding system. In the encoder side, the original video is down-sampled and compressed. In the decoder side, the decompressed down-sampled video is fed to the RR-DnCNN to get the original video by removing the compression artifacts and super-resolution. Moreover, in order to enhance the network learning capabilities, uncompressed low-resolution images/videos are utilized as a ground-truth. The experimental results show that our work can obtain over 8% BD-rate reduction compared to the standard H.265/HEVC. Furthermore, our method also outperforms in reducing compression artifacts in subjective comparison. Our work is available at https://github.com/minhmanho/rrdncnn.
Supported by JST, PRESTO Grant Number JPMJPR1757 Japan.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Shen, M., Xue, P., Wang, C.: Down-sampling based video coding using super-resolution technique. IEEE Trans. Circ. Syst. Video Technol. 21(6), 755–765 (2011)
Li, Y., et al.: Convolutional neural network-based block up-sampling for intraframe coding. IEEE Trans. Circ. Syst. Video Technol. 28(9), 2316–2330 (2018)
Lin, J., Liu, D., Yang, H., Li, H., Feng, W.: Convolutional neural network-based block up-sampling for HEVC. IEEE Trans. Circ. Syst. Video Technol. (2018)
Feng, L., Zhang, X., Zhang, X., Wang, S., Wang, R., Ma, S.: A Dual-Network Based Super-Resolution for Compressed High Definition Video. In: Hong, R., Cheng, W.-H., Yamasaki, T., Wang, M., Ngo, C.-W. (eds.) PCM 2018. LNCS, vol. 11164, pp. 600–610. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00776-8_55
Zhao, T., Zhang, C., Ren, W., Ren, D., Hu, Q.: Unsupervised degradation learning for single image super-resolution. arXiv preprint: arXiv:1812.04240 (2018)
Bulat, A., Yang, J., Tzimiropoulos, G.: To learn image super-resolution, use a GAN to learn how to do image degradation first. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018, Part VI. LNCS, vol. 11210, pp. 187–202. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_12
Chen, H., He, X., Ren, C., Qing, L., Teng, Q.: CISRDCNN: super-resolution of compressed images using deep convolutional neural networks. Neurocomputing 285, 204–219 (2018)
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)
Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep cnn for image de-noising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018, Part VII. LNCS, vol. 11211, pp. 294–310. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_18
Zhang, K., Zuo, W., Zhang, L.: Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint: arXiv:1412.6980 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ho, MM., He, G., Wang, Z., Zhou, J. (2020). Down-Sampling Based Video Coding with Degradation-Aware Restoration-Reconstruction Deep Neural Network. In: Ro, Y., et al. MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science(), vol 11961. Springer, Cham. https://doi.org/10.1007/978-3-030-37731-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-37731-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37730-4
Online ISBN: 978-3-030-37731-1
eBook Packages: Computer ScienceComputer Science (R0)