Abstract
JPEG is one of the most widely used lossy image compression algorithms, but artifacts are generated during compression. Various artifact reduction methods have been proposed, and many of them, especially deep learning-based approaches, showed promising performance. However, one major drawback that limited their deployment and application is their cumbersome and complicated model. To remedy this problem, we propose a simple and efficient network named Efficient Artifact Reduction Network. To achieve efficiency, we consider enlarging the receptive field and preserving pixel-wise information as significant concerns. On the one hand, we notice choosing a proper down-sampling ratio is important, as the down-sampling operation is a trade-off between these two aspects. On the other hand, we design a Large Kernel Depthwise Separable Convolution block that considers both aspects. For flexibility over different compression qualities, which is the focus of research in recent years, we design a Half Adaptive Instance Normalization-based approach that elegantly integrates information of the Quantization Matrix into the feature map. It adaptively normalizes half of the channels in the Encoder to embed the compression quality information and precise pixel-wise information is preserved through the other half channels. We also design a scalable architecture inspired by prior works to enable a post-training balance between computational cost and restoration performance. Experiments on various datasets show that our network achieves state-of-the-art restoration performance with much fewer parameters and less computational cost.
















Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The datasets used in this paper are public datasets and can be obtained by accessing websites or contacting the relevant providers.
Code Availability
The code of this study will be available at https://github.com/tgjjj/EARN.
References
Sheikh, H.: Live image quality assessment database release 2, http://live.ece.utexas.edu/research/quality
Wallace, G.K.: The JPEG still picture compression standard. Commun. ACM 34(4), 30–44 (1991)
Ehrlich, M., Davis, L., Lim, S.-N., Shrivastava, A.: Quantization guided jpeg artifact correction. In: European Conference on Computer Vision, pp. 293–309. Springer (2020)
Zheng, B., Chen, Y., Tian, X., Zhou, F., Liu, X.: Implicit dual-domain convolutional network for robust color image compression artifact reduction. IEEE Trans. Circuits Syst. Video Technol. 30(11), 3982–3994 (2019)
Zheng, B., Sun, R., Tian, X., Chen, Y.: S-net: a scalable convolutional neural network for JPEG compression artifact reduction. J. Electron. Imaging 27(4), 043037 (2018)
Jiang, J., Zhang, K., Timofte, R.: Towards flexible blind JPEG artifacts removal. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4997–5006 (2021)
Galteri, L., Seidenari, L., Bertini, M., Del Bimbo, A.: Deep universal generative adversarial compression artifact removal. IEEE Trans. Multimedia 21(8), 2131–2145 (2019)
Kim, Y., Soh, J.W., Park, J., Ahn, B., Lee, H.-S., Moon, Y.-S., Cho, N.I.: A pseudo-blind convolutional neural network for the reduction of compression artifacts. IEEE Trans. Circuits Syst. Video Technol. 30(4), 1121–1135 (2019)
Kim, Y., Soh, J.W., Cho, N.I.: AGARNet: adaptively gated jpeg compression artifacts removal network for a wide range quality factor. IEEE Access 8, 20160–20170 (2020)
Foi, A., Katkovnik, V., Egiazarian, K.: Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images. IEEE Trans. Image Process. 16(5), 1395–1411 (2007)
Lee, K., Kim, D.S., Kim, T.: Regression-based prediction for blocking artifact reduction in JPEG-compressed images. IEEE Trans. Image Process. 14(1), 36–48 (2004)
Koç Kayhan, S.: Efficient robust filtering technique for blocking artifacts reduction. Vis. Comput. 32(4), 417–427 (2016)
Golestaneh, S.A., Chandler, D.M.: Algorithm for JPEG artifact reduction via local edge regeneration. J. Electron. Imaging 23(1), 013018 (2014)
Dong, C., Deng, Y., Loy, C.C., Tang, X.: Compression artifacts reduction by a deep convolutional network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 576–584 (2015)
Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: European Conference on Computer Vision, pp. 184–199. Springer (2014)
Svoboda, P., Hradis, M., Barina, D., Zemcik, P.: Compression artifacts removal using convolutional neural networks. arXiv preprint arXiv:1605.00366
Cavigelli, L., Hager, P., Benini, L.: CAS-CNN: a deep convolutional neural network for image compression artifact suppression. In: 2017 International Joint Conference on Neural Networks (IJCNN), IEEE, pp. 752–759 (2017)
Fu, X., Zha, Z.-J., Wu, F., Ding, X., Paisley, J.: JPEG artifacts reduction via deep convolutional sparse coding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2501–2510 (2019)
Chen, H., He, X., Ren, C., Qing, L., Teng, Q.: CISRDCNN: super-resolution of compressed images using deep convolutional neural networks. Neurocomputing 285, 204–219 (2018)
Li, B., Shi, Y., Wang, B., Qi, Z., Liu, J.: RGSR: a two-step lossy JPG image super-resolution based on noise reduction. Neurocomputing 419, 322–334 (2021)
Amaranageswarao, G., Deivalakshmi, S., Ko, S.B.: Joint restoration convolutional neural network for low-quality image super resolution. Vis. Comput. 38, 31–50 (2022). https://doi.org/10.1007/s00371-020-019
Liu, P., Zhang, H., Zhang, K., Lin, L., Zuo, W.: Multi-level wavelet-CNN for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 773–782 (2018)
Chen, H., He, X., Qing, L., Xiong, S., Nguyen, T.Q.: DPW-SDNet: Dual pixel-wavelet domain deep CNNs for soft decoding of JPEG-compressed images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 711–720 (2018)
Florentín-Núñez, M.N., López-Rubio, E., López-Rubio, F.J.: Adaptive kernel regression and probabilistic self-organizing maps for JPEG image deblocking. Neurocomputing 121, 32–39 (2013)
Liu, X., Wu, X., Zhou, J., Zhao, D.: Data-driven sparsity-based restoration of JPEG-compressed images in dual transform-pixel domain. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5171–5178 (2015)
Wang, Z., Liu, D., Chang, S., Ling, Q., Yang, Y., Huang, T.S.: D3: deep dual-domain based fast restoration of jpeg-compressed images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2764–2772 (2016)
Guo, J., Chao, H.: Building dual-domain representations for compression artifacts reduction. In: European Conference on Computer Vision, pp. 628–644. Springer (2016)
Zhang, X., Yang, W., Hu, Y., Liu, J.: DMCNN: dual-domain multi-scale convolutional neural network for compression artifacts removal. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 390–394. IEEE (2018)
Sun, M., He, X., Xiong, S., Ren, C., Li, X.: Reduction of JPEG compression artifacts based on DCT coefficients prediction. Neurocomputing 384, 335–345 (2020)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Yim, C., Bovik, A.C.: Quality assessment of deblocked images. IEEE Trans. Image Process. 20(1), 88–98 (2011). https://doi.org/10.1109/TIP.2010.2061859
Joshi, P., Prakash, S., Rawat, S.: Continuous wavelet transform-based no-reference quality assessment of deblocked images. Vis. Comput. 34(12), 1739–1748 (2018)
Li, L., Zhou, Y., Lin, W., Wu, J., Zhang, X., Chen, B.: No-reference quality assessment of deblocked images. Neurocomputing 177, 572–584 (2016)
Nah, S., Hyun Kim, T., Mu Lee, K.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3883–3891 (2017)
Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510 (2017)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer (2015)
Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883 (2016)
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122
Zini, S., Bianco, S., Schettini, R.: Deep residual autoencoder for blind universal JPEG restoration. IEEE Access 8, 63283–63294 (2020). https://doi.org/10.1109/ACCESS.2020.2984387
Yu, F., Wang, D., Shelhamer, E., Darrell, T.: Deep layer aggregation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2403–2412 (2018)
Chen, H., He, X., Yang, H., Qing, L., Teng, Q.: A feature-enriched deep convolutional neural network for JPEG image compression artifacts reduction and its applications. IEEE Trans. Neural Netw. Learn. Syst. 33(1), 430–444 (2021)
Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2010)
Rawzor: Image compression benchmark (2022). http://imagecompression.info/
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)
Maas, A.L., Hannun, A.Y., Ng, A.Y., et al.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol. 30, , p. 3. Citeseer (2013)
Chen, H., He, X., An, C., Nguyen, T.Q.: Deep wide-activated residual network based joint blocking and color bleeding artifacts reduction for 4:2:0 JPEG-compressed images. IEEE Signal Process. Lett. 26(1), 79–83 (2019). https://doi.org/10.1109/LSP.2018.2880146
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Teng, G., Jiang, R., Liu, X. et al. EARN: toward efficient and robust JPEG compression artifact reduction. Vis Comput 40, 3033–3053 (2024). https://doi.org/10.1007/s00371-023-03008-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-023-03008-4