Skip to main content
SpringerLink
Log in

Robust face recognition against expressions and partial occlusions

  • Research Article
  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

Abstract

Facial features under variant-expressions and partial occlusions could have degrading effect on overall face recognition performance. As a solution, we suggest that the contribution of these features on final classification should be determined. In order to represent facial features’ contribution according to their variations, we propose a feature selection process that describes facial features as local independent component analysis (ICA) features. These local features are acquired using locally lateral subspace (LLS) strategy. Then, through linear discriminant analysis (LDA) we investigate the intraclass and interclass representation of each local ICA feature and express each feature’s contribution via a weighting process. Using these weights, we define the contribution of each feature at local classifier level. In order to recognize faces under single sample constraint, we implement LLS strategy on locally linear embedding (LLE) along with the proposed feature selection. Additionally, we highlight the efficiency of the implementation of LLS strategy. The overall accuracy achieved by our approach on datasets with different facial expressions and partial occlusions such as AR, JAFFE, FERET and CK+ is 90.70%. We present together in this paper survey results on face recognition performance and physiological feature selection performed by human subjects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Y. Z. Lu. A novel face recognition algorithm for distinguishing faces with various angles. International Journal of Automation and Computing, vol. 5, no. 2, pp. 193–197, 2008.

    Article  Google Scholar 

  2. A. Schwaninger, C. Wallraven, D. W. Cunningham, S. D. Chiller-Glaus. Processing of facial identity and expression: A psychophysical, physiological, and computational perspective. Progress in Brain Research, vol. 156, pp. 321–343, 2006.

    Article  Google Scholar 

  3. A. M. Martínez. Matching expression variant faces. Vision Research, vol. 43, no. 9, pp. 1047–1060, 2003.

    Article  Google Scholar 

  4. V. Bruce, A. W. Young. In the Eye of the Beholder, the Science of Face Perception, New York: Oxford University Press, 1998.

    Google Scholar 

  5. H. Hill, A. Johnston. Categorizing sex and identity from the biological motion of faces. Current Biology, vol. 11, no. 11, pp. 880–885, 2001.

    Article  Google Scholar 

  6. D. W. Jacobs, D. Weinshall, Y. Gdalyahu. Classification with nonmetric distances: Image retrieval and class representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp. 583–600, 2000.

    Article  Google Scholar 

  7. J. Ruiz-del-Solar, R. Verschae, M. Correa. Recognition of faces in unconstrained environments: A comparative study. EURASIP Journal on Advances in Signal Processing, vol. 2009, no. 15, 2009.

    Google Scholar 

  8. D. J. Beymer, T. Poggio. Face recognition from one example view. In Proceedings of the 5th International Conference on Computer Vision, IEEE, Cambridge, USA, pp. 500–507, 1995.

    Chapter  Google Scholar 

  9. T. Poggio, T. Vetter. Recognition and Structure from One 2D Model View: Observations on Prototypes, Object Classes and Symmetries, Cambridge, USA: MIT, 1992.

    Google Scholar 

  10. X. Y. Tan, S. C. Chen, Z. -H. Zhou, F. Y. Zhang. Face recognition from a single image per person: A survey. Pattern Recognition, vol. 39, no. 9, pp. 1725–1745, 2006.

    Article  MATH  Google Scholar 

  11. A. K. Jain, R. P. W. Duin, J. C. Mao. Statistical pattern recognition: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 4–37, 2000.

    Article  Google Scholar 

  12. A. J. Calder, A. W. Young. Understanding the recognition of facial identity and facial expression. Nature Reviews Neuroscience, vol. 6, no. 8, pp. 641–651, 2005.

    Article  Google Scholar 

  13. J. V. Haxby, E. A. Hoffman, M. I. Gobbini. The distributed human neural system for face perception. Trends in Cognitive Sciences, vol. 4, no. 6, pp. 223–233, 2000.

    Article  Google Scholar 

  14. V. Bruce, A. W. Young. Understanding face recognition. British Journal of Psychology, vol. 77, no. 3, pp. 305–327, 1986.

    Article  Google Scholar 

  15. D. Tranel, A. R. Damasio, H. Damasio. Intact recognition of facial expression, gender, and age in patients with impaired recognition of face identity. Neurology, vol. 38, no. 5, pp. 690–696, 1988.

    Article  Google Scholar 

  16. A. W. Young, F. Newcombe, E. H. F. de Haan, M. Small, D. C. Hay. Face perception after brain injury: Selective impairments affecting identity and expression. Brain, vol. 116, no. 4, pp. 941–959, 1993.

    Article  Google Scholar 

  17. J. Kurucz, G. Feldmar. Prosopo-affective agnosia as a symptom of cerebral organic disease. Journal of the American Geriatrics Society, vol. 27, no. 5, pp. 225–230, 1979.

    Article  Google Scholar 

  18. J. S. Winston, R. N. A. Henson, M. R. Fine-Goulden, R. J. Dolan. fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception. Journal of Neurophysiology, vol. 92, no. 3, pp. 1830–1839, 2004.

    Article  Google Scholar 

  19. K. Grill-Spector, N. Knouf, N. Kanwisher. The fusiform face area subserves face perception, not generic withincategory identification. Nature Neuroscience, vol. 7, no. 5, pp. 555–562, 2004.

    Article  Google Scholar 

  20. J. M. Kaufmann, S. R. Schweinberger. Expression influences the recognition of familiar faces. Perception, vol. 33, no. 4, pp. 399–408, 2004.

    Article  Google Scholar 

  21. J. Y. Baudouin, D. Gilibert, S. Sansone, G. Tiberghien. When the smile is a cue to familiarity. Memory, vol. 8, no. 5, pp. 285–292, 2000.

    Article  Google Scholar 

  22. J. W. Tanaka, J. A. Sengco. Features and their configuration in face recognition. Memory and Cognition, vol. 25, no. 5, pp. 583–592, 1997.

    Article  Google Scholar 

  23. P. Sinha, B. Balas, Y. Ostrovsky, R. Russell. Face recognition by humans: Nineteen results all computer vision researchers should know about. Proceedings of the IEEE, vol. 94, no. 11, pp. 1948–1962, 2006.

    Article  Google Scholar 

  24. H. D. Ellis, J. W. Shepherd, G. M. Davies. Identification of familiar and unfamiliar faces from internal and external features: Some implications for theories of face recognition. Perception, vol. 8, no. 4, pp. 431–439, 1979.

    Article  Google Scholar 

  25. Y. Andreu, R. Mollineda. The role of face parts in gender recognition. Image Analysis and Recognition, A. Campilho, M. Kamel, Eds., Berlin, Heidelberg, Germang: Springer, pp. 945–954, 2008.

  26. J. Sadr, I. Jarudi, P. Sinha. The role of eyebrows in face recognition. Perception, vol. 32, no. 3, pp. 285–293, 2003.

    Article  Google Scholar 

  27. A. W. Young, D. Hellawell, D. C. Hay. Configurational information in face perception. Perception, vol. 16, no. 6, pp. 747–759, 1987.

    Article  Google Scholar 

  28. A. M. Martinez. Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp. 748–763, 2002.

    Article  Google Scholar 

  29. B. G. Park, K. M. Lee, S. U. Lee. Face recognition using face-ARG matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1982–1988, 2005.

    Article  Google Scholar 

  30. W. H. Deng, J. N. Hu, J. Guo. Extended SRC: Undersampled face recognition via intraclass variant dictionary. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1864–1870, 2012.

    Article  Google Scholar 

  31. J. W. Lu, Y. P. Tan, G. Wang. Discriminative multimanifold analysis for face recognition from a single training sample per person. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 39–51, 2013.

    Article  Google Scholar 

  32. R. P. Wang, S. G. Shan, X. L. Chen, W. Gao. Manifold-Manifold distance with application to face recognition based on image set. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Anchorage, USA, pp. 1–8, 2008.

    Google Scholar 

  33. R. P. Wang, X. L. Chen. Manifold discriminant analysis. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Miami, USA, pp. 429–436, 2009.

    Google Scholar 

  34. X. Y. Tan, S. C. Chen, Z. H. Zhou, J. Liu. Face recognition under occlusions and variant expressions with partial similarity. IEEE Transactions on Information Forensics and Security, vol. 4, no. 2, pp. 217–230, 2009.

    Article  Google Scholar 

  35. J. W. Tanaka, M. J. Farah. Parts and wholes in face recognition. The Quarterly Journal of Experimental Psychology, vol. 46, no. 2, pp. 225–245, 1993.

    Article  Google Scholar 

  36. D. J. Beymer. Face Recognition Under Varying Pose, Center for Biological and Computational Learning, Cambridge, USA: MIT, 1993.

    Google Scholar 

  37. R. Brunelli, T. Poggio. Face recognition: Features versus templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1042–1052, 1993.

    Article  Google Scholar 

  38. A. Pentland, B. Mogghadam, T. Starner. View-based and Modular Eigenspaces for Face Recognition, Cambridge, USA: MIT, 1994.

    Book  Google Scholar 

  39. B. Heisele, P. Ho, J. Wu, T. Poggio. Face recognition: Component-based versus global approaches. Computer Vision and Image Understanding, vol. 91, no. 1–2, pp. 6–21, 2003.

    Article  Google Scholar 

  40. C. Wallraven, B. Caputo, A. Graf. Recognition with local features: The kernel recipe. In Proceedings of the 9th IEEE International Conference on Computer Vision, IEEE, Nice, France, vol. 1, pp. 257–264, 2003.

    Article  Google Scholar 

  41. W. L. Huang, H. J. Yin. On nonlinear dimensionality reduction for face recognition. Image and Vision Computing, vol. 30, no. 4–5, pp. 355–366, 2012.

    Article  MathSciNet  Google Scholar 

  42. A. M. Martinez. Recognizing expression variant faces from a single sample image per class. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, Columbus, USA, vol. 1, pp. I-353–358, 2003.

    Google Scholar 

  43. X. Y. Tan, S. C. Chen, Z. H. Zhou, F. Y. Zhang. Recognizing partially occluded, expression variant faces from single training image per person with SOM and soft k-NN ensemble. IEEE Transactions on Neural Networks, vol. 16, no. 4, pp. 875–886, 2005.

    Article  Google Scholar 

  44. T. Kanade, A. Yamada. Multi-subregion based probabilistic approach toward pose-invariant face recognition. In Proceedings of the 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation, IEEE, Kobe, Japan, vol. 2, pp. 954–959, 2003.

    Article  Google Scholar 

  45. P. N. Belhumeur, J. P. Hespanha, D. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711–720, 1997.

    Google Scholar 

  46. H. C. Kim, D. Kim, S. Y. Bang. Face recognition using LDA mixture model. Pattern Recognition Letters, vol. 24, no. 15, pp. 2815–2821, 2003.

    Article  Google Scholar 

  47. W. M. Zuo, K. Q. Wang, D. Zhang, H. Z. Zhang. Combination of two novel LDA-based methods for face recognition. Neurocomputing, vol. 70, no. 4–6, pp. 735–742, 2007.

    Article  Google Scholar 

  48. C. J. Zhou, L. Wang, Q. Zhang, X. P. Wei. Face recognition based on PCA image reconstruction and LDA. Optik-International Journal for Light and Electron Optics, vol. 124, no. 22, pp. 5599–5603, 2013.

    Article  Google Scholar 

  49. R. A. Fisher. The use of multiple measurements in taxonomic problems. Annual of Eugenics, vol. 7, no. 2, pp. 179–188, 1936.

    Article  Google Scholar 

  50. R. O. Duda, P. E. Hart. Pattern Classification and Scene Analysis, New York, USA: Wiley, 1973.

    MATH  Google Scholar 

  51. S. Lawrence, C. L. Giles, T. Ah Chung, A. D. Back. Face recognition: A convolutional neural-network approach. IEEE Transactions on Neural Networks, vol. 8, no. 1, pp. 98–113, 1997.

    Article  Google Scholar 

  52. W. Y. Yang, S. X. Liu, T. S. Jin, X. M. Xu. An optimization criterion for generalized marginal fisher analysis on undersampled problems. International Journal of Automation and Computing, vol. 8, no. 2, pp. 193–200, 2011.

    Article  Google Scholar 

  53. M. Turk, A. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71–86, 1991.

    Article  Google Scholar 

  54. S. Mika, B. Schölkopf, A. Smola, K. R. Müller, M. Scholz, G. Rätsch. Kernel PCA and de-noising in feature spaces. In Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems II, MIT, Cambridge, USA, pp. 536–542, 1999.

    Google Scholar 

  55. Y. Jian, D. Zhang, A. F. Frangi, J. Y. Yang. Twodimensional PCA: A new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 131–137, 2004.

    Article  Google Scholar 

  56. T. Cox, M. Cox. Multidimensional Scaling, London, UK: Chapman & Hall, 1994.

    MATH  Google Scholar 

  57. J. B. Tenenbaum, V. De Silva, J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, vol. 290, no. 5500, pp. 2319–2323, 2000.

    Article  Google Scholar 

  58. P. Demartines, J. Herault. Curvilinear component analysis: a self-organizing neural network for nonlinear mapping of data sets. IEEE Transactions on Neural Networks, vol. 8, no. 1, pp. 148–154, 1997.

    Article  Google Scholar 

  59. A. Hyvärinen, E. Oja. Independent component analysis: Algorithms and applications. Neural Networks, vol. 13, no. 4–5, pp. 411–430, 2000.

    Article  Google Scholar 

  60. X. Niyogi. Locality preserving projections. In Advances in Neural Information Processing Systems, MIT Press, Cambridge, USA, vol. 16, pp. 153, 2004.

    Google Scholar 

  61. S. T. Roweis, L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, vol. 290, no. 5500, pp. 2323–2326, 2000.

    Article  Google Scholar 

  62. T. Kohonen. Self-organizing Map, Berlin, Heidelberg, Germany: Springer, 1995.

    Book  MATH  Google Scholar 

  63. A. Hyvärinen. Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, vol. 10, no. 3, pp. 626–634, 1999.

    Article  Google Scholar 

  64. Y. Su, S. G. Shan, X. L. Chen, W. Gao. Hierarchical ensemble of global and local classifiers for face recognition. IEEE Transactions on Image Processing, vol. 18, no. 8, pp. 1885–1896, 2007.

    Article  MathSciNet  Google Scholar 

  65. W. G. Yi, J. Duan, M. Y. Lu. Double-layer bayesian classifier ensembles based on frequent itemsets. International Journal of Automation and Computing, vol. 9, no. 2, pp. 215–220, 2012.

    Article  Google Scholar 

  66. A. M. Martinez, R. Benavente. The AR Face Database. CVC Technical Report, [Online], Available: http://www2. ece. ohio-state. edu/ aleix/ARdatabase. html, 1998.

    Google Scholar 

  67. M. J. Lyons, J. Budynek, J. Budynek. Automatic Classification of Single Facial Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1357–1362, 1999.

    Article  Google Scholar 

  68. P. J. Phillips, H. Wechsler, J. Huang, P. J. Rauss. The FERET database and evaluation procedure for face recognition algorithms. Image and Vision Computing, vol. 16, no. 5, pp. 295–306, 1998.

    Article  Google Scholar 

  69. P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, I. Matthews. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, IEEE, San Francisco, USA, pp. 94–101, 2010.

    Chapter  Google Scholar 

  70. P. J. Phillips, M. Hyeonjoon, S. A. Rizvi, P. J. Rauss. The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090–1104, 2000.

    Article  Google Scholar 

  71. M. S. Kim, D. Kim, S. Y. Lee. Face recognition using the embedded HMM with second-order block-specific observations. Pattern Recognition, vol. 36, no. 11, pp. 2723–2735, 2003.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, Universiti Teknologi MARA, Shah Alam, 40450, Selangor, Malaysia

    Fadhlan Kamaru Zaman

  2. Department of Mechatronics Engineering, International Islamic University Malaysia, Kuala Lumpur, 50728, Malaysia

    Fadhlan Kamaru Zaman, Amir Akramin Shafie & Yasir Mohd Mustafah

Authors
  1. Fadhlan Kamaru Zaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir Akramin Shafie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yasir Mohd Mustafah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fadhlan Kamaru Zaman.

Additional information

This work was supported by Ministry of Higher EducationMalaysia, and Universiti Teknologi MARA, Malaysia

Recommended by Associate Editor De Xu

Fadhlan Kamaru Zaman received the B. Sc. (Hons.), M. Sc. and Ph. D. degrees in electrical engineering from International Islamic University, Malaysia in 2008, 2010 and 2015, respectively. He is currently attached to the Center for Computer Engineering Studies, Faculty of Electrical Engineering, Universiti Teknologi MARA, Malaysia as senior lecturer.

His research interests include surveillance system, pattern recognition, signal and image processing, artificial intelligence and computer vision.

ORCID iD: 0000-0003-1161-6452

Amir Akramin Shafie received the B.Eng. (Hons.) degree in mechanical engineering from University of Dundee, UK. He received M. Sc. degree in mechatronics from University of Abertay Dundee and Ph.D. degree in engineering from University of Dundee. From 2000 to 2005, he was a researcher in SIRIM Berhad. He is currently attached to the Department of Mechatronics Engineering, International Islamic University Malaysia, Kuala Lumpur as associate professor. He has published various articles in books, refereed journals and various international conferences, some of which have been highly cited.

His research interest includes machine vision, intelligent system and autonomous system.

Yasir Mohd Mustafah received the B.Eng. degree in electronics engineering from the University of Southampton, UK in 2004, and the Ph.D. degree from University of Queensland, Australia in 2011. He is a member of IEEE since 2009. He is currently an assistant professor in International Islamic University Malaysia.

His research interests include computer vision, signal and image processing, embedded system, intelligent system and autonomous agents.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zaman, F.K., Shafie, A.A. & Mustafah, Y.M. Robust face recognition against expressions and partial occlusions. Int. J. Autom. Comput. 13, 319–337 (2016). https://doi.org/10.1007/s11633-016-0974-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-016-0974-6

Keywords

Search

Navigation