Skip to main content

Gated Fusion of Discriminant Features for Caricature Recognition

  • Conference paper
  • First Online:
Intelligence Science and Big Data Engineering. Visual Data Engineering (IScIDE 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11935))

Abstract

Caricature recognition is a challenging problem, because there are typically geometric deformations between photographs and caricatures. It is nontrivial to learn discriminant large-margin features. To combat this challenge, we propose a novel framework by using a gated fusion of global and local discriminant features. First, we employ A-Softmax loss to jointly learn angularly discriminant features of the whole face and local facial parts. Besides, we use the convolutional block attention module (CBAM) to further boost the discriminant ability of the learnt features. Next, we use global features as dominant representation and local features as supplemental ones; and propose a gated fusion unit to automatically learn the weighting factors for these local parts and moderate local features correspondingly. Finally, an integration of all these features is used for caricature recognition. Extensive experiments are conducted on the cross-modal face recognition task. Results show that, our method significantly boosts previous state-of-the-art Rank-1 and Rank-10 from 36.27% to 55.29% and from 64.37% to 85.78%, respectively, for caricature-to-photograph (C2P) recognition. Besides, our method achieves a Rank-1 of 60.81% and Rank-10 of 89.26% for photograph-to-caricature (P2C) recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Abaci, B., Akgul, T.: Matching caricatures to photographs. SIViP 9(1), 295–303 (2015)

    Article  Google Scholar 

  2. Cao, K., Liao, J., Yuan, L.: CariGANs: unpaired photo-to-caricature translation. arXiv preprint arXiv:1811.00222 (2018)

  3. Garg, J., Peri, S.V., Tolani, H., Krishnan, N.C.: Deep cross modal learning for caricature verification and identification (cavinet). arXiv preprint arXiv:1807.11688 (2018)

  4. Hill, M.Q., et al.: Deep convolutional neural networks in the face of caricature: identity and image revealed. arXiv preprint arXiv:1812.10902 (2018)

  5. Huo, J., Gao, Y., Shi, Y., Yin, H.: Variation robust cross-modal metric learning for caricature recognition. In: Proceedings of the on Thematic Workshops of ACM Multimedia 2017, pp. 340–348. ACM (2017)

    Google Scholar 

  6. Huo, J., Li, W., Shi, Y., Gao, Y., Yin, H.: WebCaricature: a benchmark for caricature recognition. In: British Machine Vision Conference (2018)

    Google Scholar 

  7. Klare, B.F., Bucak, S.S., Jain, A.K., Akgul, T.: Towards automated caricature recognition. In: IAPR International Conference on Biometrics, pp. 139–146 (2012)

    Google Scholar 

  8. Li, W., Huo, J., Shi, Y., Gao, Y., Wang, L., Luo, J.: A joint local and global deep metric learning method for caricature recognition. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11364, pp. 240–256. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20870-7_15

    Chapter  Google Scholar 

  9. Li, W., Xiong, W., Liao, H., Huo, J., Gao, Y., Luo, J.: Carigan: caricature generation through weakly paired adversarial learning. arXiv preprint arXiv:1811.00445 (2018)

  10. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: SphereFace: deep hypersphere embedding for face recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    Google Scholar 

  11. Peng, C., Gao, X., Wang, N., Li, J.: Graphical representation for heterogeneous face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(2), 301–312 (2017)

    Article  Google Scholar 

  12. Shi, Y., Deb, D., Jain, A.K.: WarpGAN: automatic caricature generation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10762–10771 (2019)

    Google Scholar 

  13. Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: CBAM: convolutional block attention module. In: ECCV (2018)

    Google Scholar 

  14. Zhu, M., Li, J., Wang, N., Gao, X.: A deep collaborative framework for face photo-sketch synthesis. IEEE Trans. Neural Netw. Learn. Syst. 99, 1–13 (2019)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants 61601158, 61971172, 61971339, and 61702145, in part by the Education of Zhejiang Province under Grant Y201840785 and Y201942162, and in part by the China Post-Doctoral Science Foundation under Grant 2019M653563.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Fei Gao .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dai, L. et al. (2019). Gated Fusion of Discriminant Features for Caricature Recognition. In: Cui, Z., Pan, J., Zhang, S., Xiao, L., Yang, J. (eds) Intelligence Science and Big Data Engineering. Visual Data Engineering. IScIDE 2019. Lecture Notes in Computer Science(), vol 11935. Springer, Cham. https://doi.org/10.1007/978-3-030-36189-1_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-36189-1_47

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-36188-4

  • Online ISBN: 978-3-030-36189-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics