Skip to main content
Springer Nature Link
Log in

Norface: Improving Facial Expression Analysis by Identity Normalization

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15113))

Included in the following conference series:

Abstract

Facial Expression Analysis remains a challenging task due to unexpected task-irrelevant noise, such as identity, head pose, and background. To address this issue, this paper proposes a novel framework, called Norface, that is unified for both Action Unit (AU) analysis and Facial Emotion Recognition (FER) tasks. Norface consists of a normalization network and a classification network. First, the carefully designed normalization network struggles to directly remove the above task-irrelevant noise, by maintaining facial expression consistency but normalizing all original images to a common identity with consistent pose, and background. Then, these additional normalized images are fed into the classification network. Due to consistent identity and other factors (e.g. head pose, background, etc.), the normalized images enable the classification network to extract useful expression information more effectively. Additionally, the classification network incorporates a Mixture of Experts to refine the latent representation, including handling the input of facial representations and the output of multiple (AU or emotion) labels. Extensive experiments validate the carefully designed framework with the insight of identity normalization. The proposed method outperforms existing SOTA methods in multiple facial expression analysis tasks, including AU detection, AU intensity estimation, and FER tasks, as well as their cross-dataset tasks. For the normalized datasets and code please visit project page.

H. Liu and R. An—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Dataset link. https://norface-fea.github.io/.

References

  1. Ali, K., Hughes, C.E.: Facial expression recognition by using a disentangled identity-invariant expression representation. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 9460–9467. IEEE (2021)

    Google Scholar 

  2. An, R., et al.: Learning facial expression-aware global-to-local representation for robust action unit detection. Appl. Intell. 1–21 (2024)

    Google Scholar 

  3. Bai, Y., et al.: Masked autoencoders enable efficient knowledge distillers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24256–24265 (2023)

    Google Scholar 

  4. Cai, J., et al.: Identity-free facial expression recognition using conditional generative adversarial network. In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 1344–1348. IEEE (2021)

    Google Scholar 

  5. Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: Vggface2: a dataset for recognising faces across pose and age. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 67–74. IEEE (2018)

    Google Scholar 

  6. Chang, Y., Wang, S.: Knowledge-driven self-supervised representation learning for facial action unit recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20417–20426 (2022)

    Google Scholar 

  7. Chen, K., Yang, X., Fan, C., Zhang, W., Ding, Y.: Semantic-rich facial emotional expression recognition. IEEE Trans. Affect. Comput. 13(4), 1906–1916 (2022)

    Article  Google Scholar 

  8. Chen, S., Wang, J., Chen, Y., Shi, Z., Geng, X., Rui, Y.: Label distribution learning on auxiliary label space graphs for facial expression recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13984–13993 (2020)

    Google Scholar 

  9. Chen, Y., Chen, D., Wang, Y., Wang, T., Liang, Y.: CaFGraph: context-aware facial multi-graph representation for facial action unit recognition. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 1029–1037 (2021)

    Google Scholar 

  10. Chen, Y., Joo, J.: Understanding and mitigating annotation bias in facial expression recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14980–14991 (2021)

    Google Scholar 

  11. Chen, Z.M., Wei, X.S., Wang, P., Guo, Y.: Multi-label image recognition with graph convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5177–5186 (2019)

    Google Scholar 

  12. Cui, Z., Song, T., Wang, Y., Ji, Q.: Knowledge augmented deep neural networks for joint facial expression and action unit recognition. In: Advance in Neural Information Processing System, vol. 33, pp. 14338–14349 (2020)

    Google Scholar 

  13. Deng, J., Guo, J., Xue, N., Zafeiriou, S.: ArcFace: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4690–4699 (2019)

    Google Scholar 

  14. DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017)

  15. Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  16. Fan, Y., Lam, J., Li, V.: Facial action unit intensity estimation via semantic correspondence learning with dynamic graph convolution. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12701–12708 (2020)

    Google Scholar 

  17. Ge, X., et al.: Local global relational network for facial action units recognition. In: 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), pp. 01–08. IEEE (2021)

    Google Scholar 

  18. He, J., Qiu, J., Zeng, A., Yang, Z., Zhai, J., Tang, J.: FastMoE: a fast mixture-of-expert training system. arXiv preprint arXiv:2103.13262 (2021)

  19. He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)

    Google Scholar 

  20. Hong, F.T., Zhang, L., Shen, L., Xu, D.: Depth-aware generative adversarial network for talking head video generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3397–3406 (2022)

    Google Scholar 

  21. Huang, P.J., Xie, H., Huang, H.C., Shuai, H.H., Cheng, W.H.: CA-FER: mitigating spurious correlation with counterfactual attention in facial expression recognition. IEEE Trans. Affect. Comput. (2023)

    Google Scholar 

  22. Huang, W., Zhang, S., Zhang, P., Zha, Y., Fang, Y., Zhang, Y.: Identity-aware facial expression recognition via deep metric learning based on synthesized images. IEEE Trans. Multimedia 24, 3327–3339 (2021)

    Article  Google Scholar 

  23. Jacob, G.M., Stenger, B.: Facial action unit detection with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7680–7689 (2021)

    Google Scholar 

  24. Ji, Y., Hu, Y., Yang, Y., Shen, H.T.: Region attention enhanced unsupervised cross-domain facial emotion recognition. IEEE Trans. Knowl. Data Eng. (2021)

    Google Scholar 

  25. Jiang, J., Deng, W.: Disentangling identity and pose for facial expression recognition. IEEE Trans. Affect. Comput. 13(4), 1868–1878 (2022)

    Article  Google Scholar 

  26. Kim, D., Song, B.C.: Contrastive adversarial learning for person independent facial emotion recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 5948–5956 (2021)

    Google Scholar 

  27. Kim, D., Song, B.C.: Optimal transport-based identity matching for identity-invariant facial expression recognition. In: Advance in Neural Information Processing System, vol. 35, pp. 18749–18762 (2022)

    Google Scholar 

  28. Kingma, D.P., Ba, J.: ADAM: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  29. Kollias, D., Zafeiriou, S.: Affect analysis in-the-wild: Valence-arousal, expressions, action units and a unified framework. arXiv preprint arXiv:2103.15792 (2021)

  30. Lee, I., Lee, E., Yoo, S.B.: Latent-OFER: detect, mask, and reconstruct with latent vectors for occluded facial expression recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1536–1546 (2023)

    Google Scholar 

  31. Li, G., Zhu, X., Zeng, Y., Wang, Q., Lin, L.: Semantic relationships guided representation learning for facial action unit recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8594–8601 (2019)

    Google Scholar 

  32. Li, S., Deng, W.: Deep facial expression recognition: a survey. IEEE Trans. Affect. Comput. 13(3), 1195–1215 (2020)

    Article  MathSciNet  Google Scholar 

  33. Li, S., Deng, W.: A deeper look at facial expression dataset bias. IEEE Trans. Affect. Comput. 13(2), 881–893 (2020)

    Article  MathSciNet  Google Scholar 

  34. Li, S., Deng, W., Du, J.: Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2852–2861 (2017)

    Google Scholar 

  35. Li, W., Abtahi, F., Zhu, Z., Yin, L.: EAC-Net: deep nets with enhancing and cropping for facial action unit detection. IEEE Trans. Pattern Anal. Mach. Intell. 40(11), 2583–2596 (2018)

    Article  Google Scholar 

  36. Lim, J.H., Ye, J.C.: Geometric GAN. arXiv preprint arXiv:1705.02894 (2017)

  37. Liu, H., Cai, H., Lin, Q., Zhang, X., Li, X., Xiao, H.: FEDA: fine-grained emotion difference analysis for facial expression recognition. Biomed. Signal Process. Control 79, 104209 (2023)

    Article  Google Scholar 

  38. Liu, H., Cai, H., Lin, Q., Li, X., Xiao, H.: Adaptive multilayer perceptual attention network for facial expression recognition. IEEE Trans. Circ. Syst. Video Technol. 32(9), 6253–6266 (2022)

    Article  Google Scholar 

  39. Liu, H., Cai, H., Lin, Q., Li, X., Xiao, H.: Learning from more: Combating uncertainty cross-multidomain for facial expression recognition. In: Proceedings of the 31st ACM International Conference on Multimedia, pp. 5889–5898 (2023)

    Google Scholar 

  40. Liu, R., et al.: Towards a simultaneous and granular identity-expression control in personalized face generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2114–2123 (2024)

    Google Scholar 

  41. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3730–3738 (2015)

    Google Scholar 

  42. Luo, H., et al.: Normalized avatar synthesis using StyleGAN and perceptual refinement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11662–11672 (2021)

    Google Scholar 

  43. Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., Chi, E.H.: Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1930–1939 (2018)

    Google Scholar 

  44. Ma, T., Li, B., He, Q., Dong, J., Tan, T.: GaFET: learning geometry-aware facial expression translation from in-the-wild images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7115–7125 (2023)

    Google Scholar 

  45. Mallya, A., Wang, T.C., Liu, M.Y.: Implicit warping for animation with image sets. In: Advance in Neural Information Processing System, vol. 35, pp. 22438–22450 (2022)

    Google Scholar 

  46. Martinez, B., Valstar, M.F., Jiang, B., Pantic, M.: Automatic analysis of facial actions: a survey. IEEE Trans. Affect. Comput. 10(3), 325–347 (2017)

    Article  Google Scholar 

  47. Mavadati, S.M., Mahoor, M.H., Bartlett, K., Trinh, P., Cohn, J.F.: DISFA: a spontaneous facial action intensity database. IEEE Trans. Affect. Comput. 4(2), 151–160 (2013)

    Article  Google Scholar 

  48. Mollahosseini, A., Hasani, B., Mahoor, M.H.: AffectNet: a database for facial expression, valence, and arousal computing in the wild. IEEE Trans. Affect. Comput. 10(1), 18–31 (2017)

    Article  Google Scholar 

  49. Mustafa, B., Riquelme, C., Puigcerver, J., Jenatton, R., Houlsby, N.: Multimodal contrastive learning with LIMoE: the language-image mixture of experts. In: Advance in Neural Information Processing System, vol. 35, pp. 9564–9576 (2022)

    Google Scholar 

  50. Nicolle, J., Bailly, K., Chetouani, M.: Facial action unit intensity prediction via hard multi-task metric learning for kernel regression. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 6, pp. 1–6. IEEE (2015)

    Google Scholar 

  51. Niu, X., Han, H., Yang, S., Huang, Y., Shan, S.: Local relationship learning with person-specific shape regularization for facial action unit detection. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp. 11917–11926 (2019)

    Google Scholar 

  52. Ntinou, I., Sanchez, E., Bulat, A., Valstar, M., Tzimiropoulos, G.: A transfer learning approach to heatmap regression for action unit intensity estimation. IEEE Trans. Affect. Comput. 14(1), 436–450 (2021)

    Article  Google Scholar 

  53. Reed, C.J., et al.: Scale-MAE: a scale-aware masked autoencoder for multiscale geospatial representation learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4088–4099 (2023)

    Google Scholar 

  54. Ren, Y., Li, G., Chen, Y., Li, T.H., Liu, S.: PIRenderer: controllable portrait image generation via semantic neural rendering. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13759–13768 (2021)

    Google Scholar 

  55. Ruan, D., Yan, Y., Lai, S., Chai, Z., Shen, C., Wang, H.: Feature decomposition and reconstruction learning for effective facial expression recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7660–7669 (2021)

    Google Scholar 

  56. Sanchez, E., Tellamekala, M.K., Valstar, M., Tzimiropoulos, G.: Affective processes: stochastic modelling of temporal context for emotion and facial expression recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9074–9084 (2021)

    Google Scholar 

  57. Shao, Z., Liu, Z., Cai, J., Ma, L.: JAA-Net: joint facial action unit detection and face alignment via adaptive attention. Int. J. Comput. Vision 129, 321–340 (2021)

    Article  Google Scholar 

  58. Shazeer, N., et al.: Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538 (2017)

  59. She, J., Hu, Y., Shi, H., Wang, J., Shen, Q., Mei, T.: Dive into ambiguity: latent distribution mining and pairwise uncertainty estimation for facial expression recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6248–6257 (2021)

    Google Scholar 

  60. Shrout, P.E., Fleiss, J.L.: Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 86(2), 420 (1979)

    Article  Google Scholar 

  61. Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., Sebe, N.: First order motion model for image animation. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  62. Song, T., Cui, Z., Zheng, W., Ji, Q.: Hybrid message passing with performance-driven structures for facial action unit detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6267–6276 (2021)

    Google Scholar 

  63. Sun, K., et al.: High-resolution representations for labeling pixels and regions. arXiv preprint arXiv:1904.04514 (2019)

  64. Sun, X., Zeng, J., Shan, S.: Emotion-aware contrastive learning for facial action unit detection. In: 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), pp. 01–08. IEEE (2021)

    Google Scholar 

  65. Tu, C.H., Yang, C.Y., Hsu, J.Y.J.: IdenNet: identity-aware facial action unit detection. In: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pp. 1–8. IEEE (2019)

    Google Scholar 

  66. Vandenhende, S., Georgoulis, S., Van Gansbeke, W., Proesmans, M., Dai, D., Van Gool, L.: Multi-task learning for dense prediction tasks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(7), 3614–3633 (2021)

    Google Scholar 

  67. Vo, T.H., Lee, G.S., Yang, H.J., Kim, S.H.: Pyramid with super resolution for in-the-wild facial expression recognition. IEEE Access 8, 131988–132001 (2020)

    Article  Google Scholar 

  68. Wang, C., Wang, S., Liang, G.: Identity-and pose-robust facial expression recognition through adversarial feature learning. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 238–246 (2019)

    Google Scholar 

  69. Wang, J., et al.: LipFormer: high-fidelity and generalizable talking face generation with a pre-learned facial codebook. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13844–13853 (2023)

    Google Scholar 

  70. Wang, K., Peng, X., Yang, J., Lu, S., Qiao, Y.: Suppressing uncertainties for large-scale facial expression recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6897–6906 (2020)

    Google Scholar 

  71. Wang, T.C., Mallya, A., Liu, M.Y.: One-shot free-view neural talking-head synthesis for video conferencing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10039–10049 (2021)

    Google Scholar 

  72. Wang, Y., Yang, D., Bremond, F., Dantcheva, A.: Latent image animator: learning to animate images via latent space navigation. arXiv preprint arXiv:2203.09043 (2022)

  73. Wu, S., et al.: GANHead: towards generative animatable neural head avatars. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 437–447 (2023)

    Google Scholar 

  74. Wu, Z., Cui, J.: LA-Net: landmark-aware learning for reliable facial expression recognition under label noise. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20698–20707 (2023)

    Google Scholar 

  75. Xie, H.X., Lo, L., Shuai, H.H., Cheng, W.H.: Au-assisted graph attention convolutional network for micro-expression recognition. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 2871–2880 (2020)

    Google Scholar 

  76. Xie, S., Hu, H., Chen, Y.: Facial expression recognition with two-branch disentangled generative adversarial network. IEEE Trans. Circ. Syst. Video Technol. 31(6), 2359–2371 (2020)

    Article  Google Scholar 

  77. Xue, F., Wang, Q., Guo, G.: Transfer: learning relation-aware facial expression representations with transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3601–3610 (2021)

    Google Scholar 

  78. Yang, H., Ciftci, U., Yin, L.: Facial expression recognition by de-expression residue learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2168–2177 (2018)

    Google Scholar 

  79. Yang, H., Yin, L., Zhou, Y., Gu, J.: Exploiting semantic embedding and visual feature for facial action unit detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10482–10491 (2021)

    Google Scholar 

  80. Yang, H., Zhang, Z., Yin, L.: Identity-adaptive facial expression recognition through expression regeneration using conditional generative adversarial networks. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 294–301. IEEE (2018)

    Google Scholar 

  81. Yang, K., Chen, K., Guo, D., Zhang, S.H., Guo, Y.C., Zhang, W.: Face2face \(\rho \): real-time high-resolution one-shot face reenactment. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13673, pp. 55–71. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19778-9_4

    Chapter  Google Scholar 

  82. Yin, F., Zhang, Y., Cun, X., Cao, M., Fan, Y., Wang, X., Bai, Q., Wu, B., Wang, J., Yang, Y.: StyleHeat: one-shot high-resolution editable talking face generation via pre-trained StyleGAN. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13677, pp. 85–101. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19790-1_6

    Chapter  Google Scholar 

  83. You, R., Guo, Z., Cui, L., Long, X., Bao, Y., Wen, S.: Cross-modality attention with semantic graph embedding for multi-label classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12709–12716 (2020)

    Google Scholar 

  84. Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: CutMix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023–6032 (2019)

    Google Scholar 

  85. Zeng, D., Lin, Z., Yan, X., Liu, Y., Wang, F., Tang, B.: Face2Exp: combating data biases for facial expression recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20291–20300 (2022)

    Google Scholar 

  86. Zeng, H., Zhang, W., Chen, K., Zhang, Z., Li, L., Ding, Y.: Paste you into game: Towards expression and identity consistency face swapping. In: 2022 IEEE Conference on Games (CoG), pp. 1–8. IEEE (2022)

    Google Scholar 

  87. Zeng, H., Zhang, W., Chen, K., Zhang, Z., Li, L., Ding, Y.: Face identity and expression consistency for game character face swapping. Comput. Vis. Image Underst. 236, 103806 (2023)

    Article  Google Scholar 

  88. Zeng, H., et al.: FlowFace: semantic flow-guided shape-aware face swapping. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 3367–3375 (2023)

    Google Scholar 

  89. Zhang, B., et al.: MetaPortrait: identity-preserving talking head generation with fast personalized adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22096–22105 (2023)

    Google Scholar 

  90. Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)

  91. Zhang, J., et al.: FReeNet: multi-identity face reenactment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5326–5335 (2020)

    Google Scholar 

  92. Zhang, W., Guo, Z., Chen, K., Li, L., Zhang, Z., Ding, Y.: Prior aided streaming network for multi-task affective recognitionat the 2nd abaw2 competition. arXiv preprint arXiv:2107.03708 (2021)

  93. Zhang, W., Ji, X., Chen, K., Ding, Y., Fan, C.: Learning a facial expression embedding disentangled from identity. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6759–6768 (2021)

    Google Scholar 

  94. Zhang, W., Li, L., Ding, Y., Chen, W., Deng, Z., Yu, X.: Detecting facial action units from global-local fine-grained expressions. IEEE Trans. Circ. Syst. Video Technol. (2023)

    Google Scholar 

  95. Zhang, X., et al.: BP4D-spontaneous: a high-resolution spontaneous 3D dynamic facial expression database. Image Vis. Comput. 32(10), 692–706 (2014)

    Article  Google Scholar 

  96. Zhang, Y., Dong, W., Hu, B.G., Ji, Q.: Weakly-supervised deep convolutional neural network learning for facial action unit intensity estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2314–2323 (2018)

    Google Scholar 

  97. Zhang, Y., Zhao, R., Dong, W., Hu, B.G., Ji, Q.: Bilateral ordinal relevance multi-instance regression for facial action unit intensity estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7034–7043 (2018)

    Google Scholar 

  98. Zhang, Y., Yang, Q.: A survey on multi-task learning. IEEE Trans. Knowl. Data Eng. 34(12), 5586–5609 (2021)

    Article  Google Scholar 

  99. Zhang, Z., et al.: Multimodal spontaneous emotion corpus for human behavior analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3438–3446 (2016)

    Google Scholar 

  100. Zhang, Z., Li, L., Ding, Y., Fan, C.: Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3661–3670 (2021)

    Google Scholar 

  101. Zhao, J., Zhang, H.: Thin-plate spline motion model for image animation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3657–3666 (2022)

    Google Scholar 

  102. Zhao, K., Chu, W.S., Martinez, A.M.: Learning facial action units from web images with scalable weakly supervised clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2090–2099 (2018)

    Google Scholar 

  103. Zhao, K., Chu, W.S., De la Torre, F., Cohn, J.F., Zhang, H.: Joint patch and multi-label learning for facial action unit detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2207–2216 (2015)

    Google Scholar 

  104. Zhao, K., Chu, W.S., Zhang, H.: Deep region and multi-label learning for facial action unit detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3391–3399 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tongji University, Shanghai, China

    Hanwei Liu

  2. Netease Fuxi AI Lab, Hangzhou, China

    Hanwei Liu, Rudong An, Zhimeng Zhang, Bowen Ma, Wei Zhang, Yan Song, Yujing Hu & Yu Ding

  3. Hebei Agricultural University, Baoding, China

    Wei Chen

  4. School of Computer Science, Hangzhou Dianzi University, Hangzhou, China

    Yu Ding

  5. Happy Elements, Beijing, China

    Hanwei Liu, Bowen Ma & Yu Ding

Authors
  1. Hanwei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rudong An
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhimeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bowen Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yujing Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yu Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Hessen, Germany

    Stefan Roth

  4. Princeton University, Palo Alto, CA, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 19896 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, H. et al. (2025). Norface: Improving Facial Expression Analysis by Identity Normalization. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15113. Springer, Cham. https://doi.org/10.1007/978-3-031-73001-6_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-73001-6_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-73000-9

  • Online ISBN: 978-3-031-73001-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics