Cited By
View all- Singh VSharma SCuzzolin F(2024)Feature boosting with efficient attention for scene parsingNeurocomputing10.1016/j.neucom.2024.128222601(128222)Online publication date: Oct-2024
Dual Stage Semantic Information Based Generative Adversarial Network For Image Super-Resolution✱
Article No.: 15, Pages 1 - 9
Abstract
Deep learning methods for the super-resolution problem are showing great performance compared to other traditional techniques. However, these methods are unable to learn complex spatial structures and high frequency details; which leads to over-smooth results. In the present paper, a novel Generative Adversarial Network based architecture named as Residue and Semantic feature based Dual Subpixel Generative Adversarial Network has been proposed for generator and discriminator networks to solve super-resolution problem. The generator network is residue and semantic feature based dual subpixel generative architecture. This architecture is divided into two stages: premier residual stage and deuxieme residual stage. These two stages are concatenated together to form a two stage upsamping process, which enhances the feature learning capability of our model. Inter and intra residual connections are made within these two stages; helping us to sustain the high texture details of images. Semantic based information is implanted in generator to enhance the quality of objects in an image. For embedding semantic information in generator, feature maps extracted from pre-trained model are merged with the input image. To stabilize the training process, we introduced spectral normalization in the discriminator. Visual perception and mean opinion score shows that proposed method outperforms the other state-of-the-art methods.
Supplementary Material
Dual Stage Semantic Information Based Generative Adversarial Network For Image Super-Resolution
- Download
- 8.69 MB
References
[1]
Michal Aharon, Michael Elad, Alfred Bruckstein, 2006. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing 54, 11 (2006), 4311.
[2]
Pablo Arbelaez, Charless Fowlkes, and David Martin. 2007. The Berkeley segmentation dataset and benchmark. see http://www. eecs. berkeley. edu/Research/Projects/CS/vision/bsds (2007).
[3]
Moshe Ben-Ezra, Assaf Zomet, and Shree K Nayar. 2005. Video super-resolution using controlled subpixel detector shifts. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 6 (2005), 977–987.
[4]
Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie Line Alberi-Morel. 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. (2012).
[5]
Sean Borman and Robert L Stevenson. 1998. Super-resolution from image sequences-a review. In 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268). IEEE, 374–378.
[6]
Joan Bruna, Pablo Sprechmann, and Yann LeCun. 2015. Super-resolution with deep convolutional sufficient statistics. arXiv preprint arXiv:1511.05666 (2015).
[7]
Hong Chang, Dit-Yan Yeung, and Yimin Xiong. 2004. Super-resolution through neighbor embedding. In null. IEEE, 275–282.
[8]
Zhimin Chen and Yuguang Tong. 2017. Face super-resolution through wasserstein gans. arXiv preprint arXiv:1705.02438 (2017).
[9]
Zhen Cui, Hong Chang, Shiguang Shan, Bineng Zhong, and Xilin Chen. 2014. Deep network cascade for image super-resolution. In European Conference on Computer Vision. Springer, 49–64.
[10]
Shengyang Dai, Mei Han, Wei Xu, Ying Wu, Yihong Gong, and Aggelos K Katsaggelos. 2009. Softcuts: a soft edge smoothness prior for color image super-resolution. IEEE Transactions on Image Processing 18, 5 (2009), 969–981.
[11]
Hasan Demirel and Gholamreza Anbarjafari. 2011. Discrete wavelet transform-based satellite image resolution enhancement. IEEE transactions on geoscience and remote sensing 49, 6 (2011), 1997–2004.
[12]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248–255.
[13]
Emily L Denton, Soumith Chintala, arthur szlam, and Rob Fergus. 2015. Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks. In Advances in Neural Information Processing Systems, C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (Eds.). Vol. 28. Curran Associates, Inc.https://proceedings.neurips.cc/paper_files/paper/2015/file/aa169b49b583a2b5af89203c2b78c67c-Paper.pdf
[14]
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2014. Learning a deep convolutional network for image super-resolution. In European conference on computer vision. Springer, 184–199.
[15]
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2016. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence 38, 2 (2016), 295–307.
[16]
Chao Dong, Chen Change Loy, and Xiaoou Tang. 2016. Accelerating the super-resolution convolutional neural network. In European conference on computer vision. Springer, 391–407.
[17]
William T Freeman, Thouis R Jones, and Egon C Pasztor. 2002. Example-based super-resolution. IEEE Computer graphics and Applications2 (2002), 56–65.
[18]
Nik Nur Aisyah Nik Ghazali, Nazri A Zamani, Siti Norul Huda Sheikh Abdullah, and Jinjuli Jameson. 2012. Super resolution combination methods for CCTV forensic interpretation. In 2012 12th International Conference on Intelligent Systems Design and Applications (ISDA). IEEE, 853–858.
[19]
Daniel Glasner, Shai Bagon, and Michal Irani. 2009. Super-resolution from a single image. In 2009 IEEE 12th International Conference on Computer Vision (ICCV). IEEE, 349–356.
[20]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672–2680.
[21]
Hayit Greenspan. 2008. Super-resolution in medical imaging. Comput. J. 52, 1 (2008), 43–63.
[22]
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. 2017. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems. 5767–5777.
[23]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision. Springer, 694–711.
[24]
Chi-Chou Kao, Yen-Tai Lai, and Chao-Feng Tseng. 2015. Improved edge-directed super resolution. International Journal of Computers and Applications 37, 3-4 (2015), 160–167.
[25]
John A Kennedy, Ora Israel, Alex Frenkel, Rachel Bar-Shalom, and Haim Azhari. 2006. Super-resolution in PET imaging. IEEE transactions on medical imaging 25, 2 (2006), 137–147.
[26]
Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1646–1654.
[27]
Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1637–1645.
[28]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[29]
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4681–4690.
[30]
Xin Li and Michael T Orchard. 2001. New edge-directed interpolation. IEEE transactions on image processing 10, 10 (2001), 1521–1527.
[31]
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 136–144.
[32]
Wun-Ting Lin and Shang-Hong Lai. 2013. Single image super-resolution based on local self-similarity. In 2013 2nd IAPR Asian Conference on Pattern Recognition. IEEE, 191–195.
[33]
Michael Mathieu, Camille Couprie, and Yann LeCun. 2015. Deep multi-scale video prediction beyond mean square error. arXiv preprint arXiv:1511.05440 (2015).
[34]
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 (2018).
[35]
Sung Cheol Park, Min Kyu Park, and Moon Gi Kang. 2003. Super-resolution image reconstruction: a technical overview. IEEE signal processing magazine 20, 3 (2003), 21–36.
[36]
Mehdi SM Sajjadi, Bernhard Scholkopf, and Michael Hirsch. 2017. Enhancenet: Single image super-resolution through automated texture synthesis. In Proceedings of the IEEE International Conference on Computer Vision. 4491–4500.
[37]
Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1874–1883.
[38]
Jian Sun, Zongben Xu, and Heung-Yeung Shum. 2011. Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Transactions on Image Processing 20, 6 (2011), 1529–1542.
[39]
Ying Tai, Jian Yang, and Xiaoming Liu. 2017. Image super-resolution via deep recursive residual network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3147–3155.
[40]
Ying Tai, Jian Yang, Xiaoming Liu, and Chunyan Xu. 2017. Memnet: A persistent memory network for image restoration. In Proceedings of the IEEE International Conference on Computer Vision. 4539–4547.
[41]
Hiroyuki Takeda, Peyman Milanfar, Matan Protter, and Michael Elad. 2009. Super-resolution without explicit subpixel motion estimation. IEEE Transactions on Image Processing 18, 9 (2009), 1958–1975.
[42]
Wim van Aarle, Kees Joost Batenburg, Gert Van Gompel, Elke Van de Casteele, and Jan Sijbers. 2014. Super-resolution for computed tomography based on discrete tomography. IEEE Transactions on image processing 23, 3 (2014), 1181–1193.
[43]
Eric Van Reeth, Ivan WK Tham, Cher Heng Tan, and Chueh Loo Poh. 2012. Super-resolution in magnetic resonance imaging: a review. Concepts in Magnetic Resonance Part A 40, 6 (2012), 306–325.
[44]
Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. 2018. Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV). 0–0.
[45]
Daiqin Yang, Zimeng Li, and Zhenzhong Xia, Yatong andf Chen. 2015. Remote sensing image super-resolution: Challenges and approaches. In 2015 IEEE International Conference on Digital Signal Processing (DSP). IEEE, 196–200.
[46]
Jianchao Yang, John Wright, Thomas S Huang, and Yi Ma. 2010. Image super-resolution via sparse representation. IEEE transactions on image processing 19, 11 (2010), 2861–2873.
[47]
Xin Yu and Fatih Porikli. 2016. Ultra-resolving face images by discriminative generative networks. In European conference on computer vision. Springer, 318–333.
[48]
Roman Zeyde, Michael Elad, and Matan Protter. 2010. On single image scale-up using sparse-representations. In International conference on curves and surfaces. Springer, 711–730.
Index Terms
- Dual Stage Semantic Information Based Generative Adversarial Network For Image Super-Resolution✱
Recommendations
Dual stage semantic information based generative adversarial network for image super-resolution
AbstractDeep learning has revolutionized image super-resolution, yet challenges persist in preserving intricate details and avoiding overly smooth reconstructions. In this work, we introduce a novel architecture, the Residue and Semantic Feature-based ...
Highlights- RSF-DSGAN: A novel super-resolution network with integrated semantic features.
- Enhanced Detail: Fuses pre-trained features to improve image fidelity.
- Dual-Stage Upsampling: Maintains intricate details and textures.
- Stable ...
A two-channel convolutional neural network for image super-resolution
A two-channel convolutional neural network (including one shallow and one deep channel) is proposed for the single image super-resolution (SISR). Most existing methods based on convolution neural networks (CNNs) for super resolution have a shallow ...
Image Super-Resolution and Deblurring Using Generative Adversarial Network
ICCPR '19: Proceedings of the 2019 8th International Conference on Computing and Pattern RecognitionImage super-resolution and deblurring are two highly ill-posed problems that are usually dealt separately. However, real-world images are often low-resolution and have complex blurring. This paper focuses on ordinary natural scene images and ...
Comments
Information & Contributors
Information
Published In
December 2023
352 pages
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 31 January 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICVGIP '23
ICVGIP '23: Indian Conference on Computer Vision, Graphics and Image Processing
December 15 - 17, 2023
Rupnagar, India
Acceptance Rates
Overall Acceptance Rate 95 of 286 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 50Total Downloads
- Downloads (Last 12 months)45
- Downloads (Last 6 weeks)4
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
Cited By
View all- Singh VSharma SCuzzolin F(2024)Feature boosting with efficient attention for scene parsingNeurocomputing10.1016/j.neucom.2024.128222601(128222)Online publication date: Oct-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format