Abstract
In this paper, we propose a novel multi-modal mesh surface representation fusing texture and geometric data. Our approach defines an inverse mapping between different geometric descriptors computed on the mesh surface or its down-sampled version, and the corresponding 2D texture image of the mesh, allowing the construction of fused geometrically augmented images. This new fused modality enables us to learn feature representations from 3D data in a highly efficient manner by simply employing standard convolutional neural networks in a transfer-learning mode. In contrast to existing methods, the proposed approach is both computationally and memory efficient, preserves intrinsic geometric information and learns highly discriminative feature representation by effectively fusing shape and texture information at data level. The efficacy of our approach is demonstrated for the tasks of facial action unit detection, expression classification, and skin lesion classification, showing competitive performance with state of the art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abd El Meguid, M.K., et al.: Fully automated recognition of spontaneous facial expressions in videos using random forest classifiers. IEEE Trans. Affect. Comput. 5(2), 141–154 (2014). https://doi.org/10.1109/TAFFC.2014.2317711
Ballerini, L., et al.: A color and texture based hierarchical K-NN approach to theclassification of non-melanoma skin lesions. In: Celebi, M., Schaefer, G. (eds.) Color Medical Image Analysis, vol. 6, pp. 63–86. Springer, Dordrecht (2015). https://doi.org/10.1007/978-94-007-5389-1_4
Dapogny, A., et al.: Investigating deep neural forests for facial expression recognition. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG), pp. 629–633, May 2018
Huang, Y., et al.: Combining statistics of geometrical and correlative features for 3D face recognition (2006)
Kawahara, J., et al.: Deep features to classify skin lesions. In: International Symposium on Biomedical Imaging (2016)
Kittler, J., et al.: Conformal mapping of a 3D face representation onto a 2D image for CNN based face recognition. In: International Conference on Biometrics, pp. 146–155 (2018)
Li, H., et al.: 3d facial expression recognition via multiple kernel learning of multi-scale local normal patterns. In: ICPR, pp. 2577–2580 (2012)
Li, H., et al.: An efficient multimodal 2D + 3D feature-based approach to automatic facial expression recognition. Comput. Vis. Image Underst. 140(Suppl. C), 83–92 (2015)
Li, H., et al.: Multimodal 2D+3D facial expression recognition with deep fusion convolutional neural network. IEEE Trans. Multimed. 19(12), 2816–2831 (2017). https://doi.org/10.1109/TMM.2017.2713408
Ojansivu, V., Heikkilä, J.: Blur insensitive texture classification using local phase quantization. In: Elmoataz, A., Lezoray, O., Nouboud, F., Mammass, D. (eds.) ICISP 2008. LNCS, vol. 5099, pp. 236–243. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69905-7_27
Razavian, S., et al.: CNN features off-the-shelf: an astounding baseline for recognition. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 806–813 (2014)
Ross, A., Jain, A.K.: Information fusion in biometrics. Pattern Recogn. Lett. 24, 2115–2125 (2003)
Sandbach, G., et al.: Binary pattern analysis for 3D facial action unit detection. In: British Machine Vision Conference (BMVC), pp. 119.1–119.12. BMVA Press, September 2012
Sandbach, G., et al.: Local normal binary patterns for 3D facial action unit detection. In: IEEE International Conference on Image Processing (ICIP), pp. 1813–1816, September 2012
Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3D face analysis. In: Schouten, B., Juul, N.C., Drygajlo, A., Tistarelli, M. (eds.) BioID 2008. LNCS, vol. 5372, pp. 47–56. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89991-4_6
Sinha, A., Bai, J., Ramani, K.: Deep learning 3D shape surfaces using geometry images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 223–240. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_14
Yang, M., et al.: Monogenic binary pattern (MBP): a novel feature extraction and representation model for face recognition. In: International Conference on Pattern Recognition, pp. 2680–2683 (2010)
Yang, X., et al.: Automatic 3D facial expression recognition using geometric scattering representation. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 1, pp. 1–6, May 2015
Yin, L., et al.: A high-resolution 3D dynamic facial expression database. In: IEEE Conference on Face and Gesture Recognition (FG), pp. 1–6, September 2008
Zhang, W., et al.: Local gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: IEEE International Conference on Computer Vision (ICCV), vol. 1, pp. 786–791, October 2005
Zhu, X., et al.: Face alignment across large poses: a 3D solution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 146–155, June 2016
Acknowledgment
This work is supported by a research fund from Cyber-Physical Systems Center (C2PS), Khalifa University, UAE.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Taha, B., Hayat, M., Berretti, S., Werghi, N. (2020). Fused Geometry Augmented Images for Analyzing Textured Mesh. In: McDaniel, T., Berretti, S., Curcio, I., Basu, A. (eds) Smart Multimedia. ICSM 2019. Lecture Notes in Computer Science(), vol 12015. Springer, Cham. https://doi.org/10.1007/978-3-030-54407-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-54407-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-54406-5
Online ISBN: 978-3-030-54407-2
eBook Packages: Computer ScienceComputer Science (R0)