skip to main content
research-article

3D Facial Similarity Measurement and Its Application in Facial Organization

Published: 05 July 2020 Publication History

Abstract

We propose a novel framework for 3D facial similarity measurement and its application in facial organization. The construction of the framework is based on Kendall shape space theory. Kendall shape space is a quotient space that is constructed by shape features. In Kendall shape space, the shape features can be measured and is robust to similarity transformations. In our framework, a 3D face is represented by the facial feature landmarks model (FFLM), which can be regarded as the facial shape features. We utilize the geodesic in Kendall shape space to represent the FFLM similarity measurement, which can be regarded as the 3D facial similarity measurement. The FFLM similarity measurement is robust to facial expressions, head poses, and partial facial data. In our experiments, we compute the distance between different FFLMs in two public facial databases: FRGC2.0 and BosphorusDB. On average, we achieve a rank-one facial recognition rate of 98%. Based on the similarity results, we propose a method to construct the facial organization. The facial organization is a hierarchical structure that is achieved from the facial clustering by FFLM similarity measurement. Based on the facial organization, the performance of face searching in a large facial database can be improved obviously (about 400% improvement in experiments).

References

[1]
R. Ahdid, S. Safi, M. Fakir, and B. Manaut.2017. Geodesic distance on Riemannian manifold using Jacobi iterations in 3D face recognition system. Int. J. Inf. Commun. Technol. 6, 1 (2017), 10--19.
[2]
R. Ahdid, S. Safi, and B. Manaut. 2015. Three dimensional face surfaces analysis using geodesic distance. J. Comput. Sci. Applic. 3 (2015), 67--72.
[3]
B. Amberg, S. Romdhani, and T. Vetter. 2007. Optimal step nonrigid ICP algorithms for surface registration. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07). 1--8.
[4]
A. N. Ansari and M. Abdel-Mottaleb. 2005. Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition. Pattern Recog. 38, 12 (2005), 2549--2563.
[5]
S. Berretti, A. Bimbo, and P. Pala. 2010. 3D face recognition using iso-geodesic stripes. IEEE Trans. Pattern Anal. Mach. Intell. 32, 12 (2010), 2162--2177.
[6]
S. Berretti, N. Werghi, A. D. Bimbo et al. 2013. Matching 3D face scans using interest points and local histogram descriptors. Comput. Graph. 37, 5 (2013), 509--525.
[7]
S. Berretti, N. Werghi, A. D. Bimbo, et al. 2014. Selecting stable key points and local descriptors for person identification using 3D face scans. Vis. Comput. 30, 11 (2014), 1275--1292.
[8]
V. Blanz and T. Vetter. 2003. Shape analysis of elastic curves in Euclidean spaces. IEEE Trans. Pattern Anal. Mach. Intell. 25, 9 (2003), 1063--1074.
[9]
J. Booth, A. Roussos, A. Ponniah, et al. 2018. Large scale 3D morphable models. Int. J. Comput. Vis. 126, 2--4 (2018), 233--254.
[10]
A. M. Bronstein, M. M. Bronstein, R. Kimmel, et al. 2005. Three dimensional face recognition. Int. J. Comput. Vis. 64, 1 (2005), 5--30.
[11]
C. Cao, Y. Weng, K. Zhou, et al. 2014. FaceWarehouse: A 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graph. 20, 3 (2014), 413--425.
[12]
H. Chen and B. Bhanu. 2007. Human ear recognition in 3D. IEEE Trans. Pattern Anal. Mach. Intell. 29, 4 (2007), 718--737.
[13]
T. F. Cootes, G. J. Edwards, and C. J. Taylor. 2001. Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 6 (2001), 681--685.
[14]
T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham. 1995. Active shape models—their training and application. Comput. Vis. Image Underst. 61, 1 (1995), 38--59.
[15]
C. Ding and D. Tao. 2015. Robust face recognition via multimodal deep face representation. IEEE Trans. Multimedia 17, 11 (2015), 2049--2058.
[16]
C. Dorai and A. K. Jain. 1997. COSMOS-A representation scheme for 3D free-form objects. IEEE Trans. Pattern Anal. Mach. Intell. 19, 10 (1997), 1115--1130.
[17]
H. Drira, B. B. Amor, A. Srivastava, et al. 2013. 3D face recognition under expressions, occlusions, and pose variations. IEEE Trans. Pattern Anal. Mach. Intell. 35, 9 (2013), 2270--2283.
[18]
H. Drira, B. B. Amor, A. Srivastava, and M. Daoudi. 2009. A Riemannian analysis of 3D nose shapes for partial human biometrics. In Proceedings of the IEEE International Conference on Computer Vision. 2050--2057.
[19]
I. L. Dryden, A. Kume, Le Huiling, et al. 2008. A multi-dimensional scaling approach to shape analysis. Biometrika 95, 4 (2008), 779--798.
[20]
B. Efraty, E. Bilgazyev, S. Shah, et al. 2012. Profile-based 3D-aided face recognition. Pattern Recog. 45, 1 (2012), 43--53.
[21]
M. Emambakhsh and A. Evans. 2016. Nasal patches and curves for expression-robust 3D face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39, 5 (2016), 995--1007.
[22]
C. Ferrari, G. Lisanti, S. Berretti, et al. 2017. A dictionary learning based 3D morphable shape model. IEEE Trans. Multimedia 19, 12 (2017), 2666--2058.
[23]
B. J. Frey and D. Dueck. 2007. Clustering by passing messages between data points. Science 315, 5814 (2007), 972--976.
[24]
A. V. Gaikwad, S. J. Shigwan, and S. P. Awate. 2015. A statistical model for smooth shapes in Kendall shape space. In Medical Image Computing and Computer-Assisted Intervention (MICCAI'15), N. Navab, J. Hornegger, W. Wells, and A. Frangi (Eds.), Vol. 9351. 628--635.
[25]
S. Ganguly, D. Bhattacharjee, and M. Nasipuri. 2017. Fuzzy matching of edge and curvature based features from range images for 3D face recognition. Intell. Autom. Soft Comput. 23, 1 (2017), 51--62.
[26]
S. Z. Gilani and A. Mian. 2016. Towards large-scale 3D face recognition. In Proceedings of the IEEE International Conference on Digital Image Computing: Techniques and Applications. 1--8.
[27]
S. Gilani, A. Mian, and P. Eastwood. 2017. Deep, dense and accurate 3D face correspondence for generating population specific deformable models. Pattern Recog. 69 (2017), 238--250.
[28]
S. Z. Gilani, F. Shafait, and A. Mian. 2015. Shape-based automatic detection of a large number of 3D facial landmarks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4639--4648.
[29]
B. Gokberk, A. Ali Salah, and L. Akarun. 2005. Rank-based decision fusion for 3D shape-based face recognition. Audio Video-based Biomet. Person Authent. 3246 (2005), 1019--1028.
[30]
H. Hawraa, Z. Bilal, and K. Ahmed. 2019. 3D face factorisation for face recognition using pattern recognition algorithms. Cyber. Inf. Technol. 19, 2 (2019), 28--37.
[31]
D. Huang, M. Ardabilian, Y. Wang, et al. 2012. 3-D face recognition using eLBP-based facial description and local feature hybrid matching. IEEE Trans. Inf. Forens. Sec. 7, 5 (2012), 1551--1565.
[32]
S. Jahanbin, H. Choi, Y. Liu, et al. 2008. Three dimensional face recognition using iso-geodesic and iso-depth curves. In Proceedings of the IEEE International Conference on Biometrics: Theory, Applications and Systems. 1--6.
[33]
S. Jahanbin, R. Jahanbin, A. C. Bovik, et al. 2013. Passive three dimensional face recognition using iso-geodesic contours and procrustes analysis. Int. J. Comput. Vis. 105, 1 (2013), 87--108.
[34]
A. E. Johnson. 1997. Spin Images: A Representation for 3-D Surface Matching. Ph.D. Dissertation. Carnegie Mellon University, Pittsburgh, PA.
[35]
M. Kafai, K. Eshghi, and B. Bhanu. 2014. Discrete cosine transform locality-sensitive hashes for face retrieval. IEEE Trans. Multimedia 16, 4 (2014), 1090--1103.
[36]
D. G. Kendall. 1984. Shape manifolds, procrustean metrics, and complex projective spaces. Bull. London Math. Soc. 16, 2 (1984), 81--121.
[37]
P. Koppen, Z. Feng, J. Kittler, et al. 2018. Gaussian mixture 3D morphable face model. Pattern Recog. 74 (2018), 617--628.
[38]
S. Kurtek and H. Drira. 2015. A comprehensive statistical framework for elastic shape analysis of 3D faces. Comput. Graph. 51 (2015), 52--59.
[39]
Y. Lei, M. Bennamoun, M. Hayat, et al. 2014. An efficient 3D face recognition approach using local geometrical signatures. Pattern Recog. 47, 2 (2014), 509--524.
[40]
H. Li, D. Huang, J. M. Morvan, et al. 2015. Towards 3D face recognition in the real: A registration-free approach using fine-grained matching of 3D keypoint descriptors. Int. J. Comput. Vis. 113, 2 (2015), 128--142.
[41]
H. Li, J. Sun, Z. Xu, et al. 2017. Multimodal 2D+3D facial expression recognition with deep fusion convolutional neural network. IEEE Trans. Multimedia 19, 12 (2017), 2816--2831.
[42]
C. Lv, Z. Wu, X. Wang, and M. Zhou. 2019. 3D facial expression modeling based on facial landmarks in single image. Neurocomputing 355 (2019), 155--162.
[43]
C. Lv, Z. Wu, X. Wang, M. Zhou, and K. Toh. 2019. Nasal similarity measure of 3D faces based on curve shape space. Pattern Recog. 88 (2019), 458--469.
[44]
C. Lv and J. Zhao. 2018. 3D face recognition based on local conformal parameterization and iso-geodesic stripes analysis. Math. Prob. Eng. 2018 (2018), 1--10.
[45]
Z. Wang, M. Xu, Y. Ren, et al. 2018. Saliency detection in face videos: A data-driven approach. IEEE Trans. Multimedia 20, 6 (2018), 1335--1349.
[46]
S. A. Mahmood, R. F. Ghani, and A. A. Kerim. 2014. 3D face recognition using pose invariant nose region detector. In Proceedings of the Computer Science and Electronic Engineering Conference. 103--108.
[47]
I. Mpiperis, S. Malassiotis, and M. G. Strintzis. 2008. Bilinear models for 3-D face and facial expression recognition. IEEE Trans. Inf. Forens. Sec. 3, 3 (2008), 498--511.
[48]
A. Ouamane, A. Chouchane, et al. 2017. Efficient tensor-based 2D+3D face verification. IEEE Trans. Inf. Forens. Sec. 12, 11 (2017), 2751--2762.
[49]
A. Patel and W. A. P. Smith. 2015. Manifold-based constraints for operations in face space. Pattern Recog. 52 (2015), 206--217.
[50]
A. P. Pentland. 1991. Face recognition using eigenfaces. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 586--591.
[51]
P. Perakis, G. Passalis, T. Theoharis, and I. Kakadiaris. 2013. 3D facial landmark detection under large yaw and expression variations. IEEE Trans. Pattern Anal. Mach. Intell. 35, 7 (2013), 1552--1564.
[52]
P. J. Phillips, P. J. Flynn, T. Scruggs, et al. 2006. Preliminary face recognition grand challenge results. Autom. Face Gest. Recog. 15--24.
[53]
A. Savran, N. Alyuz, H. Dibeklioglu, et al. 2008. Bosphorus database for 3D face analysis. In Proceedings of the 1st COST 2101 Workshop on Biometrics and Identity Management. 47--56.
[54]
S. Schwab, T. Chateau, C. Blanc, and L. Trassoudaine. 2013. A multi-cue spatio-temporal framework for automatic frontal face clustering in video sequences. Eurasip J. Image Vid. Proc. 2013, 1 (Jan. 2013), 1--12.
[55]
D. Smeets, J. Keustermans, D. Vandermeulen, et al. 2013. MeshSIFT: Local surface features for 3D face recognition under expression variations and partial data. Comput. Vis. Image Underst. 117, 2 (2013), 158--169.
[56]
S. Soltanpour and Q. Wu. 2019. Weighted extreme sparse classifier and local derivative pattern for 3D face recognition. IEEE Trans. Image Proc. 28, 6 (2019), 3020--3033.
[57]
F. M. Sukno, J. L. Waddington, and P. F. Whelan. 2015. 3-D facial landmark localization with asymmetry patterns and shape regression from incomplete local features. IEEE Trans. Cyber. 45, 9 (2015), 1717--1730.
[58]
V. Surazhsky, T. Surazhsky, D. Kirsanov, et al. 2005. Fast exact and approximate geodesics on meshes. ACM Trans. Graph. 24, 3 (2005), 553--560.
[59]
Y. Tang, H. Li, X. Sun, et al. 2017. Principal curvature measures estimation and application to 3D face recognition. J. Math. Imag. Vis. 2017, 3 (2017), 1--23.
[60]
Y. Wang, G. Pan, Z. Wu, and Y. Wang. 2006. Exploring facial expression effects in 3D face recognition using partial ICP. In Proceedings of the Asian Conference on Computer Vision. 3851 (2006), 581--590.
[61]
P. Yan and K. W. Bowyer. 2007. Biometric recognition using 3D ear shape. IEEE Trans. Pattern Anal. Mach. Intell. 29, 8 (2007), 1297--1308.
[62]
L. Yuan, W. Liu, and Y. Li. 2016. Non-negative dictionary based sparse representation classification for ear recognition with occlusion. Neurocomputing 171, C (2016), 540--550.
[63]
W. Zeng, D. Samaras, and X. Gu. 2010. Ricci flow for 3D shape analysis. IEEE Trans. Pattern Anal. Mach. Intell. 32, 4 (2010), 662--677.
[64]
L. Zhang, L. Li, H. Li, et al. 2016. 3D ear identification using block-wise statistics-based features and LC-KSVD. IEEE Trans. Multimedia 18, 8 (2016), 1531--1541.
[65]
X. Zhu, Z. Lei, J. Yan, et al. 2015. High-fidelity pose and expression normalization for face recognition in the wild. Comput. Vis. Pattern Recog. 787--796.

Cited By

View all
  • (2024)Facial Similarity Measure for Recognizing Monozygotic Twins Utilizing 3D Facial Landmarks, Efficient Geodesic Distance Computation, and Machine Learning AlgorithmsIEEE Access10.1109/ACCESS.2024.343957212(140978-140999)Online publication date: 2024
  • (2023)KSS-ICP: Point Cloud Registration Based on Kendall Shape SpaceIEEE Transactions on Image Processing10.1109/TIP.2023.325102132(1681-1693)Online publication date: 1-Jan-2023
  • (2023)Distributed Optimization Based on Graph Filter for Ensuring Invariant Simplification of High-Volume Point CloudIEEE Transactions on Consumer Electronics10.1109/TCE.2023.327277369:3(608-621)Online publication date: 1-Aug-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 16, Issue 3
August 2020
364 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/3409646
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2020
Online AM: 07 May 2020
Accepted: 01 April 2020
Revised: 01 April 2020
Received: 01 November 2019
Published in TOMM Volume 16, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Kendall shape space
  2. face recognition
  3. facial data organization
  4. facial feature landmarks model

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)34
  • Downloads (Last 6 weeks)3
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Facial Similarity Measure for Recognizing Monozygotic Twins Utilizing 3D Facial Landmarks, Efficient Geodesic Distance Computation, and Machine Learning AlgorithmsIEEE Access10.1109/ACCESS.2024.343957212(140978-140999)Online publication date: 2024
  • (2023)KSS-ICP: Point Cloud Registration Based on Kendall Shape SpaceIEEE Transactions on Image Processing10.1109/TIP.2023.325102132(1681-1693)Online publication date: 1-Jan-2023
  • (2023)Distributed Optimization Based on Graph Filter for Ensuring Invariant Simplification of High-Volume Point CloudIEEE Transactions on Consumer Electronics10.1109/TCE.2023.327277369:3(608-621)Online publication date: 1-Aug-2023
  • (2022)Intelligent Information Network Security Management Strategy for Service Innovation of Manufacturing Enterprises Under BlockchainJournal of Organizational and End User Computing10.4018/JOEUC.31482534:1(1-17)Online publication date: 16-Sep-2022
  • (2022)Security-Aware Routing Protocol Based on Artificial Neural Network Algorithm and 6LoWPAN in the Internet of ThingsWireless Communications & Mobile Computing10.1155/2022/83744732022Online publication date: 1-Jan-2022
  • (2022)A Hybrid Convolutional Neural Network and Relief-F Algorithm for Fault Power Line Recognition in Internet of Things-Based Smart GridsWireless Communications & Mobile Computing10.1155/2022/49115532022Online publication date: 1-Jan-2022
  • (2022)Intrinsic and Isotropic Resampling for 3D Point CloudsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.3185644(1-18)Online publication date: 2022
  • (2022)RETRACTED ARTICLE: The relationship between open technological innovation, intellectual property rights capabilities, network strategy, and AI technology under the Internet of ThingsOperations Management Research10.1007/s12063-021-00242-815:3-4(793-808)Online publication date: 11-May-2022
  • (2022)Quality of Service Improvement in IoT Over Fiber-Wireless Networks Using an Efficient Routing Method Based on a Cuckoo Search AlgorithmWireless Personal Communications: An International Journal10.1007/s11277-021-09188-3126:3(2321-2346)Online publication date: 1-Oct-2022
  • (2022)An energy-aware clustering method in the IoT using a swarm-based algorithmWireless Networks10.1007/s11276-021-02804-x28:1(125-136)Online publication date: 1-Jan-2022
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media