Abstract
The images distortion leads to the loss of image information and the degradation of perceptual quality. To solve this problem, we investigate a novel mixed distortion image enhancement method based on the parallel network combines deep residual and reinforcement learning. The no-reference image quality assessment algorithm is used to determine the type and level of distorted images accurately. According to the type of distortion, the mixed distortion images enter one of the subsequent parallel joint learning networks automatically. In the joint learning framework, we prepare different residual networks to handle specialized restoration assignments including deblurring, denoising, or JPEG compression. Simultaneously, reinforcement learning then learns a policy to select the next best restoration tasks to progressively restore the quality of a corrupted image. Our method is capable of restoring images corrupted with complex mixed distortions in a more parameter-efficient manner in comparison to conventional networks. The extensive experiments on synthetic and real-world images validate the superior performances of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Dabov, K., Foi, A., Katkovnik, V., et al.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)
Krishnan, D., Tay, T., Fergus, R.: Blind deconvolution using a normalized sparsity measure[C]//Computer Vision and Pattern Recognition. IEEE, pp. 233–240 (2011)
Huibin, C., Michael, K.N., Tieyong, Z.: Reducing artifacts in JPEG decompression via a learned dictionary. IEEE Trans. Signal Process., vol. 62, no. 3, (2014)
Wang, X., Wang, Y., Xiao, Y. et al.: No-reference quality assessment method for multiply distorted images. In Packaging Engineering, 2017
Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. TPAMI 39(6), 1256–1272 (2017)
Zhang, K., Zuo, W., Chen, Y., et al.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising[J]. IEEE Trans. Image Process. 26(7), 3142–3155 (2016)
Xin, T., Hongyun, G., Xiaoyong, S., et al.: Scale-recurrent network for deep image deblurring. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8174–8182, (2018)
Nah, S., Kim, T.H., Lee, K.M.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In CVPR, (2017)
Wang, Z., Liu, D., Chang, S.: D3: Deep dual-domain based fast restoration of JPEG-compressed images. In CVPR, (2016)
Dong, C., Deng, Y., Loy, C.C. et al.: Compression artifacts reduction by a deep convolutional network. In ICCV, (2015)
Svoboda, P., Hradis, M., Barina, D., et al.: Compression artifacts removal using convolutional neural networks[J]. J. Wscg 24(2), 63–72 (2016)
Dong, C., Loy, C.C., He, K., et al.: Image super-resolution using deep convolutional networks. TPAMI 38(2), 295–307 (2016)
Wang, X., Yu, K., Dong, C., et al.: Recovering realistic texture in image super-resolution by deep spatial feature transform. In CVPR, (2018)
Gao, R., Grauman, K.: On-demand learning for deep image restoration. In IEEE International Conference on Computer Vision, vol. 11095-1104, (2017)
Zhang, K., Zuo, W., Gu, S. et al.: Learning deep cnn denoiser prior for image restoration. In CVPR, (2017)
Zhang, Y., Li, K., Li, K. et al.: Residual non-local attention networks for image restoration. In ICLR, (2019)
Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In CVPR, (2016)
Zhang, K., Zuo, W., Zhang, L., et al.: Learning a single convolutional super-resolution network for multiple degradations. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3271, (2018)
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Park, J., Lee, J.Y., Yoo, D. et al.: Distort-and-recover: color enhancement using deep reinforcement learning. In Computer Vision and Pattern Recognition, (2018)
Ke, Y., Chao, D., Liang, L., et al.: Crafting a toolchain for image restoration by deep reinforcement learning. In ICCVPR, pp. 2443–2452. IEEE, (2018)
Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: dataset and study. In CVPR Workshop, (2017)
Jayaraman, D., Mittall, A., Moorthy, A.K. et al.: Objective Quality Assessment of Multiply Distorted Images[C]//Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers, IEEE, pp. 1693–1697, (2012)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In ICLR, (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In CoRR, abs/1502.01852, 2015
Abadi, M. et al.: TensorFlow: large-scale machine learning on heterogeneous systems, Software available from tensorflow.org, (2015)
Acknowledgements
This work was supported by the Shanghai Natural Science Foundation Project (16ZR1422800) and Lab of Green Platemaking and Standardization for Flexographic Printing (ZBKT201809).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, X., Liu, F. & Ma, X. Mixed distortion image enhancement method based on joint of deep residuals learning and reinforcement learning. SIViP 15, 995–1002 (2021). https://doi.org/10.1007/s11760-020-01824-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-020-01824-y