Abstract
In this paper we propose a new family of metrics on the manifold of oriented ellipses centered at the origin in Euclidean n-space, the double cover of the manifold of positive semi-definite matrices of rank two, in order to measure similarities between landmark representations. The metrics, whose distance functions are remarkably simple, are parametrized by the choice of a n-by-n positive semi-definite matrix P. This allows us to learn the parameter P from the training data and increase the efficiency of the metric. We evaluate the proposed metric on facial expression recognition from 2D facial landmarks. The conducted experiments demonstrate the effectiveness of the learned metric to classify facial shapes under different expressions.
M. Daoudi, N. Otberdout and J.-C.Á. Paiva—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Szczapa, B., Daoudi, M., Berretti, S., Bimbo, A.D., Pala, P., Massart, E.M.: Fitting, comparison, and alignment of trajectories on positive semi-definite matrices with application to action recognition. In: 2019 IEEE/CVF International Conference on Computer Vision Workshops, ICCV Workshops 2019, Seoul, Korea (South), 27–28 October 2019, pp. 1241–1250 (2019)
Kacem, A., Daoudi, M., Ben Amor, B., Berretti, S., Alvarez-Paiva, J.C.: A novel geometric framework on Gram matrix trajectories for human behavior understanding. IEEE Trans. Pattern Anal. Mach. Intell. 42(1), 1–14 (2020)
NOtberdout, N., Kacem, A., Daoudi, M., Ballihi, L., Berretti, S.: Automatic analysis of facial expressions based on deep covariance trajectories. IEEE Trans. Neural Netw. Learn. Syst. 31, 3892–3905 (2019)
Kacem, A., Daoudi, M., Ben Amor, B., Alvarez-Paiva, J.C.: A novel space-time representation on the positive semidefinite cone for facial expression recognition. In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22–29 October 2017, pp. 3199–3208 (2017)
Abdel-Ghaffar, E.A., Daoudi, M.: Emotion recognition from multidimensional electroencephalographic signals on the manifold of symmetric positive definite matrices. In: IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) 2020, pp. 354–359 (2020)
Daoudi, M., Hammal, Z., Kacem, A., Cohn, J.F.: Gram matrices formulation of body shape motion: an application for depression severity assessment. In: 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), pp. 258–263 (2019)
Huang, Z., Wang, R., Shan, S., Chen, X.: Projection metric learning on Grassmann manifold with application to video based face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 140–149 (2015)
Zadeh, P., Hosseini, R., Sra, S.: Geometric mean metric learning. In: International Conference on Machine Learning, pp. 2464–2471 (2016)
Bonnabel, S., Sepulchre, R.: Riemannian metric and geometric mean for positive semidefinite matrices of fixed rank. SIAM J. Matrix Anal. Appl. 31(3), 1055–1070 (2010)
Massart, E.M., Hendrickx, J.M., Absil, P.-A.: Matrix geometric means based on shuffled inductive sequences. Linear Algebra Appl. 542, 334–359 (2018)
Massart, E., Hendrickx, J.M., Absil, P.-A.: Curvature of the manifold of fixed-rank positive-semidefinite matrices endowed with the Bures–Wasserstein metric. In: Nielsen, F., Barbaresco, F. (eds.) GSI 2019. LNCS, vol. 11712, pp. 739–748. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26980-7_77
Massart, E., Absil, P.-A.: Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices. SIAM J. Matrix Anal. Appl. 41(1), 171–198 (2020)
Wang, F., Zhang, C.: Feature extraction by maximizing the average neighborhood margin. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
Baltrusaitis, T., Zadeh, A., Lim, Y.C., Morency, L.-P.: OpenFace 2.0: facial behavior analysis toolkit. In: 13th IEEE International Conference on Automatic Face & Gesture Recognition, pp. 59–66 (2018)
Townsend, J., Koep, N., Weichwald, S.: Pymanopt: a python toolbox for optimization on manifolds using automatic differentiation. J. Mach. Learn. Res. 17(137), 1–5 (2016)
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J.M., Ambadar, Z., Matthews, I.A.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Conference on Computer Vision and Pattern Recognition Workshop (CVPRW), pp. 94–101 (2010)
Tanfous, A.B., Drira, H., Amor, B.B.: Sparse coding of shape trajectories for facial expression and action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 42(10), 2594–2607 (2019)
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Taheri, S., Turaga, P., Chellappa, R.: Towards view-invariant expression analysis using analytic shape manifolds. IEEE Face Gesture 2011, 306–313 (2011)
Jain, S., Hu, C., Aggarwal, J.K.: Facial expression recognition with temporal modeling of shapes. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops) 2011, pp. 1642–1649 (2011)
Wang, Z., Wang, S., Ji, Q.: Capturing complex spatio-temporal relations among facial muscles for facial expression recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3422–3429 (2013)
Szczapa, B., Daoudi, M., Berretti, S., Pala, P., Del Bimbo, A., Hammal Z.: Automatic estimation of self-reported pain by interpretable representations of motion dynamics (2020). arXiv preprint arXiv:2006.13882
Acknowledgment
The proposed work was partially supported by the French State, managed by the National Agency for Research (ANR) under the Investments for the future program with reference ANR-16-IDEX-0004 ULNE.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Daoudi, M., Otberdout, N., Paiva, JC.Á. (2021). Metric Learning on the Manifold of Oriented Ellipses: Application to Facial Expression Recognition. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12666. Springer, Cham. https://doi.org/10.1007/978-3-030-68780-9_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-68780-9_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68779-3
Online ISBN: 978-3-030-68780-9
eBook Packages: Computer ScienceComputer Science (R0)
