Kehua Guo
Skip Abstract Section
Abstract
Reference-based image super-resolution (RefSR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) images by introducing HR reference images. The key step of RefSR is to transfer reference features to LR features. However, existing methods still lack an efficient transfer mechanism, resulting in blurry details in the generated image. In this article, we propose a double-layer search module and an adaptive pooling fusion module group for reference-based image super-resolution, called DLASR. Based on the re-search strategy, the double-layer search module can produce an accurate index map and score map. These two maps are used to filter out accurate reference features, which greatly increases the efficiency of feature transfer in the later stage. Through two continuous feature-enhancement steps, the adaptive pooling fusion module group can transfer more valuable reference features to the corresponding LR features. In addition, a structure reconstruction module is proposed to recover the geometric information of the images, which further improves the visual quality of the generated image. We conduct comparative experiments on a variety of datasets, and the results prove that DLASR achieves significant improvements over other state-of-the-art methods, in terms of quantitative accuracy and qualitative visual effect. The code is available at https://github.com/clttyou/DLASR.
- [1] . 2005. Image up-sampling using total-variation regularization with a new observation model. IEEE Trans. Image Process. 14, 10 (2005), 1647–1659.Google Scholar
Digital Library
- [2] . 2014. Improving resolution and depth-of-field of light field cameras using a hybrid imaging system. In Proceedings of the 2014 IEEE International Conference on Computational Photography (ICCP). IEEE, 1–10.Google ScholarCross Ref
- [3] . 2014. Learning a deep convolutional network for image super-resolution. In Proceedings of the European Conference on Computer Vision. Springer, 184–199.Google ScholarCross Ref
- [4] . 2015. Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 2 (2015), 295–307.Google ScholarDigital Library
- [5] . 2014. Vehicle logo recognition using image matching and textural features. In Proceedings of the Scientific Cooperations International Workshops on Electrical and Computer Engineering Subfields. 82–86.Google Scholar
- [6] . 2007. Image upsampling via imposed edge statistics. In ACM SIGGRAPH 2007 Papers. 95–102.Google ScholarDigital Library
- [7] . 2015. Texture synthesis using convolutional neural networks. arXiv preprint arXiv:1505.07376 (2015).Google Scholar
- [8] . 2014. Generative adversarial networks. arXiv preprint arXiv:1406.2661 (2014).Google Scholar
- [9] . 2020. Closed-loop matters: Dual regression networks for single image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5407–5416.Google ScholarCross Ref
- [10] . 1973. Textural features for image classification. IEEE Trans. Syst Man.Cybe6 (1973), 610–621.Google ScholarCross Ref
- [11] . 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.Google ScholarCross Ref
- [12] . 2019. ATTGAN: Facial attribute editing by only changing what you want. IEEE Trans. Image Process. 28, 11 (2019), 5464–5478.Google ScholarDigital Library
- [13] . 2010. Image quality metrics: PSNR vs. SSIM. In Proceedings of the 2010 20th International Conference on Pattern Recognition. IEEE, 2366–2369.Google ScholarDigital Library
- [14] . 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4700–4708.Google ScholarCross Ref
- [15] . 2015. Single image super-resolution from transformed self-exemplars. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5197–5206.Google ScholarCross Ref
- [16] . 2021. Multi-task learning-based all-in-one collaboration framework for degraded image super-resolution. ACM Trans. Multimedia Comput. Commun. 17, 1 (2021), 1–21.Google ScholarDigital Library
- [17] . 2014. ADAM: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- [18] . 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4681–4690.Google ScholarCross Ref
- [19] . 2016. Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2414–2423.Google ScholarCross Ref
- [20] . 2020. A deep learning approach for face hallucination guided by facial boundary responses. ACM Trans. Multimedia Comput. Commun. 16, 1 (2020), 1–23.Google ScholarDigital Library
- [21] . 2001. New edge-directed interpolation. IEEE Trans. Image Process. 10, 10 (2001), 1521–1527.Google ScholarDigital Library
- [22] . 2019. Symmetrical residual connections for single image super-resolution. ACM Trans. Multimedia Comput. Commun. 15, 1 (2019), 1–10.Google ScholarDigital Library
- [23] . 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 136–144.Google ScholarCross Ref
- [24] . 2013. Network in network. arXiv preprint arXiv:1312.4400 (2013).Google Scholar
- [25] . 2020. Deepsat v2: Feature augmented convolutional neural nets for satellite image classification. Remote Sensing Letters 11, 2 (2020), 156–165.Google ScholarCross Ref
- [26] . 2021. Medical image classification based on an adaptive size deep learning model. ACM Trans. Multimedia Comput. Commun. 17, 3s (2021), 1–18.Google ScholarDigital Library
- [27] . 1999. Object recognition from local scale-invariant features. In Proceedings of the 7th IEEE International Conference on Computer Vision, Vol. 2. IEEE, 1150–1157.Google ScholarDigital Library
- [28] . 2020. SRFLOW: Learning the super-resolution space with normalizing flow. In Proceedings of the European Conference on Computer Vision. Springer, 715–732.Google ScholarDigital Library
- [29] . 2021. Understanding adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition 110 (2021), 107332.Google ScholarCross Ref
- [30] . 2017. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications 76, 20 (2017), 21811–21838.Google ScholarDigital Library
- [31] . 2003. Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine 20, 3 (2003), 21–36.Google ScholarCross Ref
- [32] . 2017. Enhancenet: Single image super-resolution through automated texture synthesis. In Proceedings of the IEEE International Conference on Computer Vision. 4491–4500.Google ScholarCross Ref
- [33] . 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1874–1883.Google ScholarCross Ref
- [34] . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- [35] . 2008. Image super-resolution using gradient profile prior. In Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1–8.Google Scholar
- [36] . 2010. Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans. Image Process. 20, 6 (2010), 1529–1542.Google Scholar
- [37] . 2018. ESRGAN: Enhanced super-resolution generative adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops.Google Scholar
- [38] . 2015. Single image superresolution based on gradient profile sharpness. IEEE Trans. Image Process. 24, 10 (2015), 3187–3202.Google ScholarDigital Library
- [39] . 2020. Learning texture transformer network for image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5791–5800.Google ScholarCross Ref
- [40] . 2017. Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans. Image Process. 26, 12 (2017), 5895–5907.Google ScholarDigital Library
- [41] . 2013. Landmark image super-resolution by retrieving web images. IEEE Trans. Image Process. 22, 12 (2013), 4865–4878.Google ScholarDigital Library
- [42] . 2010. On single image scale-up using sparse-representations. In Proceedings of the International Conference on Curves and Surfaces. Springer, 711–730.Google ScholarDigital Library
- [43] . 2020. Kernel attention network for single image super-resolution. ACM Trans. Multimedia Comput. Commun. 16, 3 (2020), 1–15.Google ScholarDigital Library
- [44] . 2006. An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans. Image Process. 15, 8 (2006), 2226–2238.Google ScholarDigital Library
- [45] . 2019. Ranksrgan: Generative adversarial networks with ranker for image super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 3096–3105.Google ScholarCross Ref
- [46] . 2018. Image super-resolution using very deep residual channel attention networks. In Proceedings of the European Conference on Computer Vision (ECCV). 286–301.Google ScholarDigital Library
- [47] . 2018. Residual dense network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2472–2481.Google ScholarCross Ref
- [48] . 2019. Image super-resolution by neural texture transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7982–7991.Google ScholarCross Ref
- [49] . 2018. Cross-scale reference-based light field super-resolution. IEEE Trans. Comput. Imaging 4, 3 (2018), 406–418.Google ScholarCross Ref
- [50] . 2017. Learning cross-scale correspondence and patch-based synthesis for reference-based super-resolution. In Proceedings of the BMVC.Google ScholarCross Ref
- [51] . 2018. Crossnet: An end-to-end reference-based super resolution network using cross-scale warping. In Proceedings of the European Conference on Computer Vision (ECCV). 88–104.Google ScholarDigital Library
- [52] . 2020. Efficient spatio-temporal recurrent neural network for video deblurring. In Proceedings of the European Conference on Computer Vision. Springer, 191–207.Google ScholarDigital Library
- [53] . 2021. Cross view capture for stereo image super-resolution. IEEE Trans. Multimedia (2021).Google Scholar
- [54] . 2021. Lightweight image super-resolution with expectation-maximization attention mechanism. IEEE Trans. Circuits Syst. Video Techn. (2021).Google Scholar
- [55] . 2015. Modeling deformable gradient compositions for single-image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5417–5425.Google ScholarCross Ref
Index Terms
- Double-Layer Search and Adaptive Pooling Fusion for Reference-Based Image Super-Resolution
Recommendations
Image super-resolution: use of self-learning and gabor prior
ACCV'12: Proceedings of the 11th Asian conference on Computer Vision - Volume Part IIIRecent approaches on single image super-resolution (SR) have attempted to exploit self-similarity to avoid the use of multiple images. In this paper, we propose an SR method based on self-learning and Gabor prior. Given a low resolution (LR) test image ...
Hyperspectral image super-resolution through clustering-based sparse representation
AbstractPromoting the spatial resolution of hyperspectral sensors is expected to improve computer vision tasks. However, due to the physical limitations of imaging sensors, the hyperspectral image is often of low spatial resolution. In this paper, we ...
Fast Learning-Based Single Image Super-Resolution
We present a learning-based single image super-resolution (SISR) method to obtain a high resolution (HR) image from a single given low resolution (LR) image. Our method gives more accurate results while also testing (runs) and training faster with a ...
Comments