Skip to main content
SpringerLink
Log in

Pooled shrinkage estimator for quadratic discriminant classifier: an analysis for small sample sizes in face recognition

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The quadratic discriminant classifier (QDC) is a well-known parametric Bayesian classifier that has been successfully applied to statistical pattern recognition problems. One such application is in automatic face recognition where the number of training images per subject is often found to be much less than the length of the facial features. In such a case, the QDC cannot be used because the class-specific covariance matrix on which it depends is either poorly estimated or singular thereby resulting in unacceptable classifier performance. High dimensional covariance estimation techniques such as shrinkage can alleviate this problem but only to a certain extent. This paper presents a computationally simple yet effective solution for further improving the QDC performance in small sample size scenarios. The proposed technique adopts a strategy of combining the class-specific shrinkage estimates of the covariance matrix to obtain a pooled shrinkage estimate, which is then plugged into the QDC. Experiments indicate that the proposed classifier leads to remarkable improvement in face recognition accuracy as compared to the existing classifiers such as the nearest neighbor, support vector machine and naive Bayes, irrespective of the nature of the database and feature extraction method. Monte Carlo simulations reveal that this improvement is due to the much lower mean squared error of the pooled shrinkage estimator which offers greater stability to the QDC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Ahonen T, Hadid A (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  Google Scholar 

  2. Ali SS, Howlader T, Rahman SMM (2014) Quadratic discriminant classifier for Gauss-Hermite moment-based face recognition. In: Proceedings of the ISI regional statistical conference, pp 1–12. Kuala Lumpur, Malaysia, Nov 16–19 2014. International Statistical Institute

  3. Bai J, Shi S (2011) Estimating high dimensional covariance matrices and its applications. Ann Econ Fin 12(2):199–215

    MathSciNet  Google Scholar 

  4. Belhumeur PN, Hespanha JP, Kreigman DJ (1997) Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  5. Chen BC, Chen CS, Hsu WH (2014) Cross-age reference coding for age-invariant face recognition and retrieval. In: Proceedings of the European conference on computer vision, pp 768–783. Zurich, Switzerland

  6. Chen Y, Wiesel A, Eldar YC, Hero AO (2010) Shrinkage algorithms for MMSE covariance estimation. IEEE Trans Signal Process 58(10):5016–5029

    Article  MathSciNet  Google Scholar 

  7. DeMiguel V, Martin-Utrera A (2013) Size matters: optimal calibration of shrinkage estimators for portfolio selection. J Bank Finan 37(8):3018–3034

    Article  Google Scholar 

  8. Friedman JH (1989) Regularized discriminant analysis. J Am Stat Assoc 84(405):165–175

    Article  MathSciNet  Google Scholar 

  9. Hoffbeck JP, Landgrebe DA (1996) Covariance matrix estimation and classification with limited training data. IEEE Trans Pattern Anal Mach Intell 18(7):763–767

    Article  Google Scholar 

  10. Huang GB, Ramesh M, Berg T, Miller EL (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst

  11. Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37

    Article  Google Scholar 

  12. Janzamin M, Anandkumar A (2014) High-dimensional covariance decomposition into sparse Markov and independence models. J Mach Learn Res 15(Apr):1549–1591

    MathSciNet  MATH  Google Scholar 

  13. Kanade T, Cohn JF, Tian YL (2000) Comprehensive database for facial expression analysis. In: Proceedings of the IEEE international conference on automatic face and gesture recognition, pp 484–490. Grenoble, France, Mar 28–30, 2000. IEEE Biometrics Council

  14. Kittler J (1994) Statistical pattern recognition in image analysis. J Appl Stat 21(1–2):61–75

    Article  Google Scholar 

  15. Kwan CCY (2008) Estimation error in the average correlation of security returns and shrink- age estimation of covariance and correlation matrices. Finan Res Lett 5(1):236–244

    Article  MathSciNet  Google Scholar 

  16. Ledoit O, Wolf M (2003) Improved estimation of the covariance matrix of stock returns with an application to portfolio selection. J Emp Finan 10(5):1–20

    Google Scholar 

  17. Ledoit O, Wolf M (2004) A well-conditioned estimator for large-dimensional covariance matrices. J Multivar Anal 88:365–411

    Article  MathSciNet  MATH  Google Scholar 

  18. Ledoit O, Wolf M (2012) Nonlinear shrinkage estimation of large-dimensional covariance matrices. Ann Stat 40(2):1024–1060

    Article  MathSciNet  MATH  Google Scholar 

  19. Li M, Yuan B (2005) 2D-LDA: a statistical linear discriminant analysis for image matrix. Pattern Recogn Lett 26(5):527–532

    Article  Google Scholar 

  20. Lu J, Plataniotis KN, Venetsanopoulos AN (2003) Regularized discriminant analysis for the small sample size problem in face recognition. Pattern Recogn Lett 24(16):3079–3087

    Article  Google Scholar 

  21. Lu SX, Wang XZ, Zhang GQ, Zhou X (2015) Effective algorithms of the Moore-Penrose inverse matrices for extreme learning machine. Intell Data Anal 19(4):743–760

    Article  Google Scholar 

  22. Mitra S, Lazar NA, Liu Y (2007) Understanding the role of facial asymmetry in human face identification. Stat Comput 17(1):57–70

    Article  MathSciNet  Google Scholar 

  23. Phillips PJ, Flynn PJ, Scruggs T, Bowyer KW, Chang J, Hoffman K, Marques J, Min J, Worek W (2005) Overview of the face recognition grand challenge. In: Proceedings of the IEEE international conference on computer vision and pattern recognition, pp 947–954, San Diego, CA, Jun 20–26, 2005. IEEE Computer Society

  24. Pourahmadi M (2011) Covariance estimation: the GLM and regularization perspectives. Ann Econ Finan 26(3):369–387

    MathSciNet  MATH  Google Scholar 

  25. Pourahmadi M (2013) High-dimensional covariance estimation: with high-dimensional data, 1st edn. Wiley, NJ

    Book  MATH  Google Scholar 

  26. Rahman SMM, Lata SP, Howlader T (2015) Bayesian face recognition using 2D Gaussian-Hermite moments. EURASIP J Image Video Process 2015(35):1–20

    Google Scholar 

  27. Rivera AR, Castillo JR, Chae O (2013) Local directional number pattern for face analysis: face and expression recognition. IEEE Trans Image Process 22(5):1740–1752

    Article  MathSciNet  MATH  Google Scholar 

  28. Schafer J, Strimmer K (2005) A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat Appl Genet Mol Biol 4(1):1544–6115

    Article  MathSciNet  Google Scholar 

  29. Sharma A, Paliwal KK (2015) Linear discriminant analysis for the small sample size problem: an overview. Int J Mach Learn Cybern 6(3):443–454

    Article  Google Scholar 

  30. Shen J, Shen W, Shen D (2000) On geometric and orthogonal moments. Int J Pattern Recogn Artif Intell 14(7):875–894

    Article  MATH  Google Scholar 

  31. Srivastava S, Gupta MR, Frigyik BA (2007) Bayesian quadratic discriminant analysis. J Mach Learn Res 8(Jun):1277–1305

    MathSciNet  MATH  Google Scholar 

  32. Stein C (1975) Estimation of a covariance matrix. In: Rietz Lecture, Atlanta, GA, 1975. 39th annual meeting

  33. Sun Y, Chen Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. In: Proceedings of advances in neural information processing systems 27, pp 1988–1996, Montreal, Canada, Dec 8–13, 2014. Curran Associates, Inc

  34. Thomaz CE, Gillies DF, Feitosa RQ (2004) A new covariance estimate for Bayesian classifiers in biometric recognition. IEEE Trans Circuits Syst Video Technol 14(2):214–223

    Article  Google Scholar 

  35. Tian YL, Kanade T, Cohn JF (2001) Recognizing action units for facial expression analysis. IEEE Trans Pattern Anal Mach Intell 23(2):97–115

    Article  Google Scholar 

  36. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  37. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  38. Wang C, Pan G, Tong T, Zhu L (2015a) Shrinkage estimation of large dimensional precision matrix using random matrix theory. Stat Sin 25(3):993–1008

    MathSciNet  MATH  Google Scholar 

  39. Wang XZ (2015) Learning from big data with uncertainty editorial. J Intell Fuzzy Syst 28(5):2329–2330

    Article  MathSciNet  Google Scholar 

  40. Wang XZ, Ashfaq RAR, Fu AM (2015b) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196

    Article  MathSciNet  Google Scholar 

  41. Won J, Lim J, Kim S, Rajaratnam B (2013) Condition-number-regularized covariance estimation. J R Stat Soc B 75(3):427–450

    Article  MathSciNet  Google Scholar 

  42. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  43. Yang B, Dai M (2011) Image analysis by Gaussian-Hermite moments. Signal Process 91(10):2290–2303

    Article  MATH  Google Scholar 

  44. Yang B, Li G, Zhang H, Dai M (2011) Rotation and translation invariants of Gaussian-Hermite moments. Pattern Recogn Lett 32(9):1283–1298

    Article  Google Scholar 

  45. Yang J, Zhang D, Frangi AF, Yang J-Y (2004) Two-dimensional PCA: a new approach of appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137

    Article  Google Scholar 

  46. Yang M, Zhu PF, Liu F, Shen LL (2015) Joint representation and pattern learning for robust face recognition. Neurocomputing 168:70–80

    Article  Google Scholar 

  47. Yao J, Chang C, Salmi ML, Hung YS, Loraine A, Roux SJ (2008) Genome-scale cluster analysis of replicated microarrays using shrinkage correlation coefficient. BMC Bioinf 9(288):1–16

    Google Scholar 

Download references

Acknowledgments

The authors would like to give thanks to the anonymous reviewers for their valuable comments that were useful to improve the quality of the paper.

Author information

Authors and Affiliations

  1. Institute of Statistical Research and Training, Dhaka University, Dhaka, 1000, Bangladesh

    Syed Shahnewaz Ali & Tamanna Howlader

  2. Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology, Dhaka, 1205, Bangladesh

    S. M. Mahbubur Rahman

Authors
  1. Syed Shahnewaz Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tamanna Howlader
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. M. Mahbubur Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. M. Mahbubur Rahman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ali, S.S., Howlader, T. & Rahman, S.M.M. Pooled shrinkage estimator for quadratic discriminant classifier: an analysis for small sample sizes in face recognition. Int. J. Mach. Learn. & Cyber. 9, 507–522 (2018). https://doi.org/10.1007/s13042-016-0549-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-016-0549-4

Keywords

Search

Navigation