Fusing Normal Vector and Curvature Features on the Mesh for 3D Facial Expression Recognition

3D Facial Expression Recognition (FER) has received increasing attention in various applications (e.g., human-computer interaction and driver fatigue detection), as it can enable the machines to understand human intentions or emotions more accurately. However, the current use of 2D projection geometric features has presented limitations when distinguishing facial expressions. To address the above problem, we propose a novel solution by fusing normal vector and curvature features on a mesh. More specifically, we are the first to extract the azimuth and elevation information obtained by the triangle facet normal projection (i.e., mesh-AE descriptor). To the best of our knowledge, we are the first to fuse normal and curvature information on a 3D mesh rather than using 2D projected facial attribute maps for 3D FER. Then, we use the proposed two types of features (mesh-AE descriptor and mesh-H descriptor) to train a two-channel convolutional neural network. Principal Component Analysis (PCA) has greatly increased the efficiency of training. Finally, a linear Support Vector Machine (SVM) is used to identify six types of facial expressions. The experimental results demonstrate the well-designed system can realize accurate and generalized 3D FER.