Han-Yuan Chang
ABSTRACT
The JPEG codec is still the dominant and common image compression format despite the continuous development of new image compression techniques. In order to raise the compression ratio, distortion caused by some quantization operation within the encoding process is unavoidable. Therefore, this paper proposed a deep learning based JPEG decoder to decrease image quality degradation by JPEG encoding quantization. Most of the image processing problems, such as image denoising, image super-resolution, and even style conversion, can be handled well in the spatial domain by deep learning using the fully convolutional networks (FCN). However, because the image is converted from the spatial domain to the frequency domain by the Discrete Cosine Transform (DCT) during JPEG encoding, this paper addresses the issue of FCN might not be suitable for processing frequency-spatial domain directly. Then, this study develops two learning models more suitable for inverse DCT (IDCT) with fully connected layers and transposed convolutional layers than the results with FCN. Finally, we constructed a JPEG decoder model based on deep residual networks. The experimental results showed that the FCN does not handle the frequency domain data well, and our proposed decoder has shown better results in peak signal to noise ratio (PSNR), structural similarity index (SSIM), and multi-scale SSIM (MS-SSIM) than the general JPEG decoder, giving evidence that the learning based JPEG decoder is capable of restoring some of the image quality degradation caused by JPEG quantization.
- Simonyan, K., & Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.Google Scholar
- Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. 2017, February. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence.Google ScholarDigital Library
- Anwar, S., & Barnes, N. 2019. Real image denoising with feature attention. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 3155-3164).Google ScholarCross Ref
- Chen, L., Lu, X., Zhang, J., Chu, X., & Chen, C. 2021. HINet: Half instance normalization network for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 182-192).Google ScholarCross Ref
- Brock, A., Donahue, J., & Simonyan, K. 2018. Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096.Google Scholar
- Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. 2020. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456.Google Scholar
- Girshick, R. 2015. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440-1448).Google ScholarDigital Library
- Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. 2020. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.Google Scholar
- Teranishi, R., Goto, T., & Nagata, T. 2020. Improvement of Robustness Blind Image Restoration Method Using Failing Detection Process. Journal of Image and Graphics, 8(3), 85-92.Google ScholarCross Ref
- Dong, C., Loy, C. C., He, K., & Tang, X. 2015. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2), 295-307.Google Scholar
- Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., ... & Shi, W. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4681-4690).Google ScholarCross Ref
- Lim, B., Son, S., Kim, H., Nah, S., & Mu Lee, K. 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 136-144).Google ScholarCross Ref
- Izumi, Y., Suto, D., Kawakami, S., & Kudo, H. 2022. Blind Image Restoration and Super-Resolution for Multispectral Images Using Sparse Optimization. Journal of Image and Graphics.Google ScholarCross Ref
- Ballé, J., Laparra, V., & Simoncelli, E. P. 2016. End-to-end optimized image compression. arXiv preprint arXiv:1611.01704.Google Scholar
- Mentzer, F., Toderici, G. D., Tschannen, M., & Agustsson, E. 2020. High-fidelity generative image compression. Advances in Neural Information Processing Systems, 33, 11913-11924.Google Scholar
- Joint Photographic Experts Group 2004. JPEG standards: ISO/IEC IS 10918-1, ITU-T Recommendation T.81.Google Scholar
- Ahmed, N., Natarajan, T., & Rao, K. R. 1974. Discrete cosine transform. IEEE transactions on Computers, 100(1), 90-93.Google Scholar
- Busson, A. J. G., Mendes, P. R. C., de S. Moraes, D., da Veiga, Á. M. G., Colcher, S., & Guedes, Á. L. V. 2020, November. Decoder-Side Quality Enhancement of JPEG Images Using Deep Learning-Based Prediction Models for Quantized DCT Coefficients. In Proceedings of the Brazilian Symposium on Multimedia and the Web (pp. 129-136).Google Scholar
- Ballé, J., Laparra, V., & Simoncelli, E. P. 2015. Density modeling of images using a generalized normalization transformation. arXiv preprint arXiv:1511.06281.Google Scholar
- Agustsson, E., & Timofte, R. 2017. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 126-135).Google ScholarCross Ref
- Kingma, D. P., & Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.Google Scholar
- https://www.twcc.ai/Google Scholar
- https://github.com/hanyuan97/learned-jped-decoderGoogle Scholar
Index Terms
- Deep Learning Approach to the Quality Restoration for JPEG Images
Recommendations
Decoder-Side Quality Enhancement of JPEG Images Using Deep Learning-Based Prediction Models for Quantized DCT Coefficients
WebMedia '20: Proceedings of the Brazilian Symposium on Multimedia and the WebMany recent works have successfully applied some types of Convolutional Neural Networks (CNNs) to reduce the noticeable distortion resulting from the lossy JPEG compression technique. Most of them are built upon the processing made on the spatial ...
An Approach for Color Image Compression of JPEG and PNG Images Using DCT and DWT
CICN '14: Proceedings of the 2014 International Conference on Computational Intelligence and Communication NetworksNow a days image compression has become is an indispensible part of digitized image storage and transmission. Compression of an image is necessary before storing and transmitting it due to its limitation of storage and bandwidth capacity. Wavelet ...
Entropy-constrained predictive residual vector quantization of digital images
ICIP '95: Proceedings of the 1995 International Conference on Image Processing (Vol. 3)-Volume 3 - Volume 3A major problem with a VQ based image compression scheme is its codebook search complexity. Recently, a new VQ scheme called predictive residual vector quantizer (PRVQ) was proposed by Rizvi and Nasrabadi (see Proc. IEEE Int. Conf. Image Processing (...
Comments