skip to main content
10.1145/3641584.3641675acmotherconferencesArticle/Chapter ViewAbstractPublication PagesaiprConference Proceedingsconference-collections
research-article

Image Super-Resolution Based on Variational Autoencoder and Channel Attention

Published: 14 June 2024 Publication History

Abstract

Super-resolution (SR) method based on generative adversarial networks (GANs) has achieved excellent performance in both visual perception and image quality. However, there is still room for improvement. Therefore, we propose a variational autoencoder (VAE) network architecture. The VAE encoder can learn the probability distribution of the low-resolution (LR) image and reflect the probability with a latent variable, and the decoder restores the original image through latent variables. The VAE and discriminator work together to effectively distinguish between generated images and real high-resolution (HR) images. In addition, we introduce a channel attention (CA) mechanism into the discriminator to improve the cohesion between channels and extract useful features more effectively. With the help of VAE and CA, the proposed method achieves not only higher peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) values, but also more realistic visual quality. The experimental results verify the feasibility of the proposed method.

References

[1]
Z. Yang, P. Shi and D. Pan, “A Survey of Super-Resolution Based on Deep Learning,” 2020 International Conference on Culture-oriented Science & Technology (ICCST), Beijing, China, 2020, pp. 514-518.
[2]
Z. Wang, J. Chen and S. C. H. Hoi, “Deep Learning for Image Super-Resolution: A Survey,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 10, pp. 3365-3387, 1 Oct. 2021.
[3]
H. Greenspan, “Super-resolution in medical imaging,” The Compute Journal, vol. 52, Issue 1, January 2009, pp.43–63. https://doi.org/10.1093/comjnl/bxm075
[4]
Y. Huang, L. Shao and A. F. Frangi, “Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images Using Weakly-Supervised Joint Convolutional Sparse Coding,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 5787-5796.
[5]
L. Zhang, H. Zhang, H. Shen, and P. Li, “A super-resolution reconstruction algorithm for surveillance images,” Elsevier Signal Process., vol. 90, pp. 848–859, 2010. https://doi.org/10.1016/j.sigpro.2009.09.002
[6]
Rasti, P., Uiboupin, T., Escalera, S., Anbarjafari, G. (2016). Convolutional Neural Network Super Resolution for Face Recognition in Surveillance Monitoring. In: Perales, F., Kittler, J. (eds) Articulated Motion and Deformable Objects. AMDO 2016. Lecture Notes in Computer Science(), vol 9756. Springer, Cham. https://doi.org/10.1007/978-3-319-41778-3_18
[7]
L. Wang, K. Lu and P. Liu, “Compressed Sensing of a Remote Sensing Image Based on the Priors of the Reference Image,” in IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 4, pp. 736-740, April 2015.
[8]
Dong, C., Loy, C.C., He, K., Tang, X. (2014). Learning a Deep Convolutional Network for Image Super-Resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8692. Springer, Cham. https://doi.org/10.1007/978-3-319-10593-2_13
[9]
J. Kim, J. K. Lee and K. M. Lee, “Accurate Image Super-Resolution Using Very Deep Convolutional Networks,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 1646-1654.
[10]
Zhang Y, Li K, Wang L, Zhong B, Fu Y. Image super-resolution using very deep residual channel attention networks [C]. Proceedings of the European Conference on Computer Vision(ECCV). Munich, Germany: Springer,2018.286-301. https://doi.org/10.48550/arXiv.1807.02758
[11]
C. Ledig et al., “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 105-114.
[12]
W. Ruangsang and S. Aramvith, “Efficient super-resolution algorithm using overlapping bicubic interpolation,” 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE), Nagoya, Japan, 2017, pp. 1-2.
[13]
Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In ECCVW, 2018. 1, 2, 5, 6, 12. https://doi.org/10.48550/arXiv.1809.00219
[14]
X. Wang, L. Xie, C. Dong and Y. Shan, “Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data,” 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada, 2021, pp. 1905-1914.
[15]
J. Liang, H. Zeng and L. Zhang, “Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 5647-5656.
[16]
W. Li, Kun. Zhou, Lu. Qi, L. Lu, N. Jiang, J. Lu, J. Jia, “Best-Buddy GANs for Highly Detailed Image Super-Resolution,” arXiv preprint arXiv:2103.15295, 2021. https://doi.org/10.48550/arXiv.2103.15295
[17]
Y. Chen, S. Liu and X. Wang, “Learning Continuous Image Representation with Local Implicit Image Function,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp. 8624-8634.
[18]
M. Tancik et al., “Learned Initializations for Optimizing Coordinate-Based Neural Representations,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp. 2845-2854.
[19]
L. Wang, Y. Wang, Z. Lin, J. Yang, W. An and Y. Guo, “Learning A Single Network for Scale-Arbitrary Super-Resolution,” 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 4781-4790.
[20]
S. Son and K. M. Lee, “SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp. 7778-7787.
[21]
J. Hu, L. Shen, S. Albanie, G. Sun and E. Wu, “Squeeze-and-Excitation Networks,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 8, pp. 2011-2023, 1 Aug. 2020.
[22]
Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. CBAM: Convolutional block attention module. In ECCV, 2018. https://doi.org/10.48550/arXiv.1807.06521
[23]
Z. -S. Liu, W. -C. Siu and Y. -L. Chan, “Photo-Realistic Image Super-Resolution via Variational Autoencoders,” in IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 4, pp. 1351-1365, April 2021.
[24]
D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” in Proc. Int. Conf. Learn. Represent. (ICLR), Dec. 2014, pp. 1–14. https://doi.org/10.48550/arXiv.1312.6114
[25]
H. Basak, R. Kundu, A. Agarwal and S. Giri, “Single Image Super-Resolution using Residual Channel Attention Network,” 2020 IEEE 15th International Conference on Industrial and Information Systems (ICIIS), RUPNAGAR, India, 2020, pp. 219-224.
[26]
Z. -S. Liu, W. -C. Siu and L. -W. Wang, “Variational AutoEncoder for Reference based Image Super-Resolution,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA, 2021, pp. 516-525.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
AIPR '23: Proceedings of the 2023 6th International Conference on Artificial Intelligence and Pattern Recognition
September 2023
1540 pages
ISBN:9798400707674
DOI:10.1145/3641584
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Channel attention
  2. Discriminator
  3. Image super-resolution
  4. Variational autoencoder

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

AIPR 2023

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 21
    Total Downloads
  • Downloads (Last 12 months)21
  • Downloads (Last 6 weeks)4
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media