Abstract
Convolutional neural networks (CNNs) have been widely applied to computer vision fields due to its excellent performance. CNN-based single image super resolution (SR) methods are also put into practice and outperform previous methods. In this paper, we propose a coarse-to-fine CNN method to boost the existing CNN-based SR methods. We design a cascaded CNN architecture with three stages. The first stage takes the low-resolution (LR) image as the input and outputs a high-resolution (HR) image, then the next stage similarly takes the high-resolution result as the input and produces a finer HR image. Finally, the last stage can obtain the finest HR image. Our architecture is trained as one entire CNN which combines three loss functions to optimize the gradient descent procedure. Experiments on ImageNet-based training samples validates the effectiveness of our method on the public benchmark datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Irani, M., Peleg, S.: Improving resolution by image registration. Graph. Models Image Process. 53(3), 231–239 (1991)
Wang, H., Gao, X., Zhang, K., Li, J.: Single image super-resolution using Gaussian process regression with dictionary-based sampling and student-t likelihood. IEEE Trans. Image Process. 26(7), 3556–3568 (2017)
Bevilacqua, M., Roumy, A., Guillemot, C., Alberimorel, M.: Low complexity single-image super-resolution based on nonnegative neighbor embedding. In: British Machine Vision Conference, Guildford, Surrey, UK (2012)
Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)
Camponez, M.O., Salles, E.O.T., Sarcinellifilho, M.: Super-resolution image reconstruction using nonparametric Bayesian INLA approximation. IEEE Trans. Image Process. 21(8), 3491–3501 (2012)
Zeng, K., Yu, J., Wang, R., Li, C., Tao, D.: Coupled deep auto-encoder for single image super-resolution. IEEE Trans. Cybern. 47(1), 27–37 (2015)
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)
Zhou, F., Yang, W., Liao, Q.: Interpolation-based image super-resolution using multisurface fitting. IEEE Trans. Image Process. 21(7), 3312–3318 (2012)
Tai, W., Liu, S., Brown, M.S., Lin, S.: Super resolution using edge prior and single image detail synthesis. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2400–2407. IEEE, San Francisco, CA, USA (2010)
Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)
Chang, H., Yeung, D., Xiong, Y.: Super-resolution through neighbor embedding. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1063–6919. IEEE, Washington, DC, USA (2004)
Lai, W.-S., Huang, J.-B., Ahuja, N., Yang, M.-H.: Deep Laplacian pyramid networks for fast and accurate super-resolution. arXiv:1704.03915 [cs.CV]
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, USA, pp. 1097–1105 (2012)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 28th IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Boston, MA, USA (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A.P., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-realistic single image super-resolution using a generative adversarial network. In: 28th IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Las Vegas, NV, USA (2016)
Goodfellow, I.J., Pougetabadie, J., Mirza, M., Xu, B., Wardefarley, D., Ozair, S., Courville, A.C., Bengio, Y.: Generative adversarial nets. In: International Conference on Neural Information Processing Systems, Montréal, Canada, pp. 2672–2680 (2014)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 5th International Conference on Learning Representations, San Diego, California, USA (2015)
Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: 29th IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654. IEEE, Las Vegas, Nevada, USA (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 29th IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE, Las Vegas, Nevada, USA (2016)
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)
Bevilacqua, M., Roumy, A., Guillemot, C., Alberimorel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In 23rd British Machine Vision Conference, Guildford, Surrey, UK, pp. 1–10 (2012)
Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. Curves Surf. 6920, 711–730 (2012)
Martin, D.R., Fowlkes, C.C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: International Conference on Computer Vision, Vancouver, Canada, pp. 416–423 (2001)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: 27th International Conference on Machine Learning, Haifa, Israel, pp. 807–814 (2010)
Yangqing, J., Evan, S., Jeff, D., Sergey, K., Jonathan, L., Ross, G., Sergio, G., Trevor, D.: Caffe: convolutional architecture for fast feature embedding. In: 22nd ACM Multimedia, Orlando, Florida, USA, pp. 675–678 (2014)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 4(13), 600–612 (2004)
Timofte, R., De Smet, V., Van Gool, L.: A+: adjusted anchored neighborhood regression for fast super-resolution. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 111–126. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16817-3_8
Huang, J., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: 28th IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Boston, MA, USA (2015)
Acknowledgement
The research was supported by National Natural Science Foundation of China (61671332), Basic Research Program of Shenzhen City (JCYJ20170306171431656) and Hubei Province Technological Innovation Major Projects (2017AAA123, 2016AAA015).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Zhou, L., Wang, Z., Wang, S., Luo, Y. (2018). Coarse-to-Fine Image Super-Resolution Using Convolutional Neural Networks. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10705. Springer, Cham. https://doi.org/10.1007/978-3-319-73600-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-73600-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73599-3
Online ISBN: 978-3-319-73600-6
eBook Packages: Computer ScienceComputer Science (R0)