Abstract
Recently, facial priors have been widely used to improve the quality of super-resolution (SR) facial images, but it is underutilized in existing methods. On the one hand, facial priors such as semantic maps may be inaccurately estimated on low-resolution (LR) images or low-scale feature maps with \(L_{1}\) loss. On the other hand, it is inefficient to guide SR features with constant prior knowledge via concatenation at only one intermediate layer of the guidance network. In this paper, we focus on face super-resolution (FSR) based on semantic maps guidance and propose two simple and efficient designs to address the above two limitations respectively. In particular, to address the first limitation, we propose a novel one-hot supervision strategy to pursue accurate semantic maps, which focuses more on penalizing misclassified pixels by relaxing the regression constraint. In addition, a semantic progressive guidance network (SPGN) is proposed that uses semantic maps to learn modulation parameters in normalization layers to efficiently guide SR features layer by layer. Extensive experiments on two benchmark datasets show that the proposed method improves the state-of-the-art in both quantitative and qualitative results at \(\times \)8 scale.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bai, Y., Zhang, Y., Ding, M., Ghanem, B.: Finding tiny faces in the wild with generative adversarial network. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 21–30 (2018)
Chen, L., Su, H., Ji, Q.: Face alignment with kernel density deep neural network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6992–7002 (2019)
Kumar, A., et al.: LUVLi Face alignment: estimating landmarks location, uncertainty, and visibility likelihood. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8236–8246 (2020)
Masi, I., Mathai, J., AbdAlmageed, W.: Towards learning structure via consensus for face segmentation and parsing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5508–5518 (2020)
Pan, J., Ren, W., Hu, Z., Yang, M.H.: Learning to deblur images with exemplars. IEEE Trans. Patt. Anal. Mach. Intell 41(6), 1412–1425 (2019)
Ge, S., Zhao, S., Li, C., Zhang, Y., Li, J.: Efficient low-resolution face recognition via bridge distillation. IEEE Trans. Image Process. 29, 6898–6908 (2020)
Ge, S., Zhao, S., Gao, X., Li, J.: Fewer-shots and lower-resolutions: towards ultrafast face recognition in the wild. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 229–237 (2019)
Hsu, C.C., Lin, C.W., Su, W.T., Cheung, G.: Sigan: siamese generative adversarial network for identity-preserving face hallucination. IEEE Trans. Image Process. 28, 6225–6236 (2019)
Hong, S., Ryu, J.: Unsupervised face domain transfer for low-resolution face recognition. IEEE Signal Process. Lett. 27, 156–160 (2019)
Zhu, S., Liu, S., Loy, C.C., Tang, X.: Deep cascaded Bi-network for face hallucination. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 614–630. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_37
Yu, X., Fernando, B., Ghanem, B., Porikli, F., Hartley, R.: Face super-resolution guided by facial component heatmaps. In: Proceedings of the European Conference on Computer Vision, pp. 217–233 (2018)
Bulat, A., Tzimiropoulos, G.: Super-fan: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with gans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 109–117 (2018)
Kim, D., Kim, M., Kwon, G., Kim, D.S.: Progressive face super-resolution via attention to facial landmark. arXiv preprint arXiv:1908.08239 (2019)
Chen, Y., Tai, Y., Liu, X., Shen, C., Yang, J.: Fsrnet: end-to-end learning face super-resolution with facial priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2492–2501 (2018)
Wang, C., Zhong, Z., Jiang, J., Zhai, D., Liu, X.: Parsing map guided multi-scale attention network for face hallucination. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2518–2522 (2020)
Hu, X., et al.: Face super-resolution guided by 3D facial priors. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12349, pp. 763–780. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_44
Yin, Y., Robinson, J., Zhang, Y., Fu, Y.: Joint super-resolution and alignment of tiny faces. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12693–12700 (2020)
Xin, J., Wang, N., Gao, X., Li, J.: Residual attribute attention network for face image super-resolution. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 9054–9061 (2019)
Ma, C., Jiang, Z., Rao, Y., Lu, J., Zhou, J.: Deep face super-resolution with iterative collaboration between attentive recovery and landmark estimation. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 5569–5578 (2020)
Shen, Z., Lai, W. S., Xu, T., Kautz, J., Yang, M.H.: Deep semantic face deblurring. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 8260–8269 (2018)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)
Kim, J., Lee, J. K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision, pp. 286–301 (2018)
Yu, X., Porikli, F.: Ultra-resolving face images by discriminative generative networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 318–333. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_20
Park, T., Liu, M.Y., Wang, T.C., Zhu, J.Y.: Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 2337–2346 (2019)
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of the IEEE Conference on Computer Vision, pp. 3730–3738 (2015)
Le, V., Brandt, J., Lin, Z., Bourdev, L., Huang, T.S.: Interactive facial feature localization. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7574, pp. 679–692. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33712-3_49
Luo, L., Xue, D., Feng, X.: Ehanet: an effective hierarchical aggregation network for face parsing. Appl. Sci. 10(9), 3135 (2020)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Jang, E., Gu, S., Poole, B.: Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016)
Liu, Z.S., Siu, W.C., Chan, Y.L.: Reference based face super-resolution. IEEE. Access 7, 129112–129126 (2019)
Acknowledgement
This research was supported partially by National Nature Science Foundation of China (U1903214, 62072347, 62071338, 61876135), in part by the Nature Science Foundation of Hubei under Grant (2018CFA024, 2019CFB472), in part by Hubei Province Technological Innovation Major Project (No. 2018AAA062).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, J., Chen, J., Wang, Z., Liang, C., Han, Z., Lin, CW. (2022). Face Super-Resolution with Better Semantics and More Efficient Guidance. In: Magnenat-Thalmann, N., et al. Advances in Computer Graphics. CGI 2022. Lecture Notes in Computer Science, vol 13443. Springer, Cham. https://doi.org/10.1007/978-3-031-23473-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-23473-6_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23472-9
Online ISBN: 978-3-031-23473-6
eBook Packages: Computer ScienceComputer Science (R0)