Skip to main content
SpringerLink
Log in

Face quality analysis of single-image super-resolution based on SIFT

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Single-image super-resolution (SISR) aims at improving image quality, and there so far exist many SISR algorithms to hallucinate super-resolution (super-res) image from simulated low-res image. In order to evaluate SISR algorithms, objective image quality assessment (IQA), e.g., full reference IQA and no-reference IQA, and subjective quality are usually estimated. However, the objective IQA usually does not well match with the subjective quality. This paper therefore introduces a new measurement based on SIFT key-points. Both descriptors and locations of SIFT key-points are used to detect the matched SIFT key-points between super-res image and its high-res label image. The more the matched SIFT key-points are, the closer super-res image should be to its high-res label image, that is the SISR algorithm is able to recover more SIFT key-points. Both simulated low-res faces and real low-res face are employed to validate the evaluation strategy. The normalization of the number of SIFT key-points is proposed and mean opinion score from 30 raters are collected to evaluate SISR algorithms. The experimental results show that the objective IQA based on SIFT key-points are able to effectively evaluate SISR algorithms, and can well match with the subjective IQA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Hu, X., Peng, S., Wang, L., Yang, Z., Li, Z.: Surveillance video face recognition with single sample per person based on 3D modeling and blurring. Neurocomputing 235, 46–58 (2017)

    Article  Google Scholar 

  2. Yang, Z., Hu, X., Dai, F., Pang, J., Jiang, T., Tao, D.: Person re-identification by discriminant analytical least squares metric learning. Mach. Vis. Appl. 217(2), 301–333 (2018)

    Google Scholar 

  3. Biswas, S., Aggarwal, G., Flynn, P.J., Bowyer, K.W.: Pose-robust recognition of low-resolution faces. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 3037–3049 (2013)

    Article  Google Scholar 

  4. Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multi frame super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2014)

    Article  Google Scholar 

  5. Zhu, S., Li, Y.: Single image super-resolution under multi-frame method. SIViP 13, 331–339 (2019)

    Article  MathSciNet  Google Scholar 

  6. Liu, C., Shum, H.Y., Freeman, W.T.: Face hallucination: theory and practice. Int. J. Comput. Vis. 75(1), 115–134 (2007)

    Article  Google Scholar 

  7. Wang, X., Tang, X.: Hallucinating face by eigentransformation. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 35(3), 425–434 (2005)

    Article  Google Scholar 

  8. Yang, J., Wright, J.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  Google Scholar 

  9. Fanaee, F., Yazdi, M., Faghihi, M.: Face image super-resolution via sparse representation and wavelet transform. SIViP 13, 79–86 (2019)

    Article  Google Scholar 

  10. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  11. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1646–1654 (2016)

  12. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2808–2817 (2017)

  13. Ledig, C., Theis, L., Huszár, F., Caballero, J. Cunningham, A., Acosta, A. Aitken, A. Tejani, A. Totz, J. Wang, Z., Twitter, W.S.: Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105–114 (2017)

  14. Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Loy, C.C., Qiao, Y., Tang, X.: ESRGAN: Enhanced super-resolution generative adversarial networks. In: European Conference on Computer Vision (ECCV), pp. 1–17 (2018)

  15. Bulat, A., Tzimiropoulos, G.: Super-FAN: Integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with GANs. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–14 (2018)

  16. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  17. Farah, J., Hojeij, M.R., Chrabieh, J., Dufaux, F.: Full-reference and reduced-reference quality metrics based on SIFT. In: IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP), pp. 161–165 (2014)

  18. Wu, Q., Li, H., Meng, F., Ngan, K.N., Luo, B., Huang, C., Zeng, B.: Blind image quality assessment based on multichannel feature fusion and label transfer. IEEE Trans. Circuits Syst. Video Technol. 26(3), 425–440 (2016)

    Article  Google Scholar 

  19. Rakhshanfar, M., Amer, M.A.: Sparsity-based no-reference image quality assessment for automatic denoising. SIViP 12, 739–747 (2018)

    Article  Google Scholar 

  20. Li, J., Yan, J., Deng, D., Shi, W., Deng, S.: No-reference image quality assessment based on hybrid model. SIViP 11, 985–992 (2017)

    Article  Google Scholar 

  21. Lowe, D.G.: Distinctive image features from scale invariant keypoints. Int. J. Comput. Vis. 2(60), 91–110 (2004)

    Article  Google Scholar 

  22. Abdel-Hakim, A.E., Farag, A.A.: CSIFT: a SIFT descriptor with color invariant characteristics. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1978–1983 (2006)

  23. Bay, H., Tuytelaars, T., Goo, L.V.: SURF: speeded up robust features. In: European Conference on Computer Vision (ECCV), pp. 404–417 (2006)

  24. Maeng, H., Liao, S., Kang, D., Lee, S.W., Jain, A.K.: Nighttime face recognition at long distance cross-distance and cross-spectral matching. Pattern Recognit. 47(12), 3750–3766 (2014)

    Article  Google Scholar 

  25. Grgic, M., Delac, K., Grgic, S.: SCface–surveillance cameras face database. Multimed. Tools Appl. 51, 863–879 (2011)

    Article  Google Scholar 

  26. Wu, Q., Li, H., Meng, F., Ngan, K.N.: A perceptually weighted rank correlation indicator for objective image quality assessment. IEEE Trans. Image Process. 27(5), 2499–2513 (2018)

    Article  MathSciNet  Google Scholar 

  27. Bosse, S., Maniry, D., Mller, K.R., Wiegand, T., Samek, W.: Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27(1), 206–219 (2018)

    Article  MathSciNet  Google Scholar 

  28. Hu, X., Ma, P., Mai, Z., Peng, S., Yang, Z., Wang, L.: Face hallucination from low quality images using definition-scalable inference. Pattern Recognit. 94, 110–121 (2019)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou, 510080, China

    Xiao Hu, Juan Sun, Zhuohao Mai, Shuyi Li & Shaohu Peng

Authors
  1. Xiao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhuohao Mai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuyi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaohu Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao Hu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, X., Sun, J., Mai, Z. et al. Face quality analysis of single-image super-resolution based on SIFT. SIViP 14, 829–837 (2020). https://doi.org/10.1007/s11760-019-01614-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01614-1

Keywords

Search

Navigation