Abstract
Recently, many Convolutional Neural Network (CNN) algorithms have been proposed for image super-resolution, but most of them aim at architecture or natural scene images. In this paper, we propose a new fractal residual network model for face image super-resolution, which is very useful in the domain of surveillance and security. The architecture of the proposed model is composed of multi-branches. Each branch is incrementally cascaded with multiple self-similar residual blocks, which makes the branch appears as a fractal structure. Such a structure makes it possible to learn both global residual and local residual sufficiently. We propose a multi-scale progressive training strategy to enlarge the image size and make the training feasible. We propose to combine the loss of face attributes and face structure to refine the super-resolution results. Meanwhile, adversarial training is introduced to generate details. The results of our proposed model outperform other benchmark methods in qualitative and quantitative analysis.
The work is supported by the National Natural Science Foundation of China under Grant No.: 61976132 and the National Natural Science Foundation of Shanghai under Grant No.: 19ZR1419200.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vis. Appl. 25(6), 1423–1468 (2014). https://doi.org/10.1007/s00138-014-0623-4
Chao, D., Chen, C.L., Kaiming, H., Xiaoou, T.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)
Christian, L., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)
Jiwon, K., Jung, K.L., Kyoung, M.L.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)
Yang, W., et al.: Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans. Image Process. 26(12), 5895–5907 (2017)
Wei-Sheng, L., Jia-Bin, H., Narendra, A., Ming-Hsuan, Y.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 624–632 (2017)
Duchon, C.E.: Lanczos filtering in one and two dimensions. J. Appl. Meteor. 18(8), 1016–1022 (1979)
Zhang, L., Xiaolin, W.: An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans. Image Process. 15(8), 2226–2238 (2006)
Kim, K.I., Kwon, Y.: Single-image super-resolution using sparse regression and natural image prior. IEEE Trans. Pattern Anal. Mach. Intell. 32(6), 1127–1133 (2010)
Elad, M., Feuer, A.: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)
Hu, H., Lisimachos, P.K.: An image super-resolution algorithm for different error levels per frame. IEEE Trans. Image Process. 15(3), 592–603 (2006)
Zhang, K., Gao, X., Tao, D., Li, X.: Single image super-resolution with non-local means and steering kernel regression. IEEE Trans. Image Process. 21(11), 4544–4556 (2012)
Yu, C., Ying, T., Xiaoming, L., Chunhua, S., Jian, Y.: FSRNET: end-to-end learning face super-resolution with facial priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2492–2501 (2018)
Bee, L., Sanghyun, S., Heewon, K., Seungjun, N., Kyoung, M.L.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)
Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 9, 1167–1183 (2002)
Adrian, B., Georgios, T.: Super-fan: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with gans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 109–117 (2018)
Le, V., Brandt, J., Lin, Z., Bourdev, L., Huang, T.S.: Interactive facial feature localization. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7574, pp. 679–692. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33712-3_49
Ziwei, L., Ping, L., Xiaogang, W., Xiaoou, T.: Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3730–3738 (2015)
Oriol , V., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, pp. 3630–3638 (2016)
Deokyun, K., Minseon, K., Gihyun, K., Dae-Shik, K.: Progressive face super-resolution via attention to facial landmark. arXiv preprint arXiv:1908.08239 (2019)
Zhen, L., Jinglei, Y., Zheng, L., Xiaomin, Y., Gwanggil, J., Wei, W.: Feedback network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3867–3876 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Fang, Y., Ran, Q., Li, Y. (2020). Fractal Residual Network for Face Image Super-Resolution. In: Farkaš, I., Masulli, P., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2020. ICANN 2020. Lecture Notes in Computer Science(), vol 12396. Springer, Cham. https://doi.org/10.1007/978-3-030-61609-0_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-61609-0_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61608-3
Online ISBN: 978-3-030-61609-0
eBook Packages: Computer ScienceComputer Science (R0)