Skip to main content
SpringerLink
Log in

Dual sparse learning via data augmentation for robust facial image classification

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Data augmentation has been utilized to improve the accuracy and robustness of face recognition algorithms. However, most of the previous studies focused on using the augmentation techniques to enlarge the feature set, while the diversity produced by the virtual samples lacked sufficient attention. In sparse dictionary learning-based face recognition, \(l_1\)-based sparse representation (SR) and SVD-based dictionary learning (DL) both have shown promising performance. How to utilize both of them in an enhanced training process by data augmentation is still unclear. This paper proposes a novel method that utilizes the sample diversity generated by data augmentation and integrates sparse representation with dictionary learning, to learn dual sparse features for robust face recognition. An additional feature set is created by applying sample augmentation via simply horizontal flipping of face images. The two sparse models, \(l_1\)-based SR and SVD-based DL, are integrated together using our new proposed objective function. Under two-level fusion of both data and classifiers, the diversity between two training sets is well learned and utilized, in three implementations, to obtain a robust face recognition. After conducting extensive experiments on some popular facial datasets, we demonstrate the proposed method can produce a higher classification accuracy than many state-of-the-art algorithms, and it can be considered as a promising option for image-based face recognition. Our code is released at GitHub.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1.  The code to extract deep features using FaceNet via TensorFlow - https://github.com/zengsn/facenet.

  2.  The code to extract deep features using ResNet via TensorFlow - https://github.com/zengsn/TF_FeatureExtraction.

References

  1. Aharon M, Elad M, Bruckstein AM (2006) K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    MATH  Google Scholar 

  2. Arigbabu OA, Ahmad SMS, Adnan WAW, Yussof S (2015) Integration of multiple soft biometrics for human identification. Pattern Recognit Lett 68:278–287

    Google Scholar 

  3. Biggio B, Melis M, Fumera G, Roli F (2015) Sparse support faces. In: 2015 international conference on biometrics (ICB), IEEE, pp 208–213

  4. Bloice MD, Stocker C, Holzinger A (2017) Augmentor: an image augmentation library for machine learning. arXiv preprint arXiv:1708.04680

  5. Boult T, Cruz S, Dhamija A, Gunther M, Henrydoss J, Scheirer W (2019) Learning and the unknown: Surveying steps toward open world recognition. In: Proceedings of the AAAI conference on artificial intelligence, vol. 33, pp 9801–9807

  6. Cai S, Zhang L, Zuo W, Feng X (2016) A probabilistic collaborative representation based approach for pattern classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2950–2959

  7. Candemir S, Borovikov E, Santosh K, Antani S, Thoma G (2015) Rsilc: rotation-and scale-invariant, line-based color-aware descriptor. Image Vis Comput 42:1–12

    Google Scholar 

  8. Cao F, Hu H, Lu J, Zhao J, Zhou Z, Wu J (2016) Pose and illumination variable face recognition via sparse representation and illumination dictionary. Knowl Based Syst 107:117–128

    Google Scholar 

  9. Chen B, Li J, Ma B, Wei G (2018) Discriminative dictionary pair learning based on differentiable support vector function for visual recognition. Neurocomputing 272:306–313

    Google Scholar 

  10. Chen L, Man H, Nefian AV (2005) Face recognition based on multi-class mapping of fisher scores. Pattern Recognit 38(6):799–811

    Google Scholar 

  11. Cho S, Cha K (1996) Evolution of neural network training set through addition of virtual samples. In: Proceedings of IEEE international conference on evolutionary computation, IEEE, pp 685–688

  12. Donoho DL, Tsaig Y, Drori I, Starck JL (2012) Sparse solution of underdetermined systems of linear equations by stagewise orthogonal matching pursuit. IEEE Trans Inf Theory 58(2):1094–1121

    MathSciNet  MATH  Google Scholar 

  13. Du Y, Wang Y (2016) Generating virtual training samples for sparse representation of face images and face recognition. J Mod Opt 63(6):536–544

    MathSciNet  Google Scholar 

  14. Fawzi A, Samulowitz H, Turaga D, Frossard P (2016) Adaptive data augmentation for image classification. In: IEEE international conference on image processing (ICIP), pp 3688–3692

  15. Han B, He B, Sun T, Yan T, Ma M, Shen Y, Lendasse A (2016) Hsr: L1/2-regularized sparse representation for fast face recognition using hierarchical feature selection. Neural Comput Appl 27(2):305–320

    Google Scholar 

  16. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  17. He Z, Patel V (2017) Sparse representation-based open set recognition. IEEE Trans Pattern Anal Mach Intell 39(8):1690–1696

    Google Scholar 

  18. Huang GB, Learned-Miller E (2014) Labeled faces in the wild: updates and new reporting procedures. University of Massachusetts, Amherst, Technical Report UM-CS-2014-003, pp 1–5

  19. Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07–49, University of Massachusetts, Amherst

  20. Hussain MF, Wang H, Santosh K (2018) Gray level face recognition using spatial features. In: International conference on recent trends in image processing and pattern recognition, Springer, pp 216–229

  21. Inoue H (2018) Data augmentation by pairing samples for images classification. arXiv preprint arXiv:1801.02929

  22. Jiang Z, Lin Z, Davis LS (2013) Label consistent k-svd: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal Mach Intell 35(11):2651–2664

    Google Scholar 

  23. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, vol 25. Curran Associates Inc, Lake Tahoe, pp 1097–1105

    Google Scholar 

  24. Learned-Miller E, Huang G, RoyChowdhury A, Li H, Hua G (2016) Labeled faces in the wild: a survey. In: Advances in face detection and facial image analysis. Springer, pp 189–248

  25. Li Z, Lai Z, Xu Y, Yang J, Zhang D (2015) A locality-constrained and label embedding dictionary learning algorithm for image classification. IEEE Trans Neural Netw Learn Syst 28(2):278–293

    MathSciNet  Google Scholar 

  26. Lu Z, Zhang L (2016) Face recognition algorithm based on discriminative dictionary learning and sparse representation. Neurocomputing 174:749–755

    Google Scholar 

  27. Lv JJ, Shao XH, Huang JS, Zhou XD, Zhou X (2017) Data augmentation for face recognition. Neurocomputing 230:184–196

    Google Scholar 

  28. Marqués I, Graña M (2013) Fusion of lattice independent and linear features improving face identification. Neurocomputing 114:80–85

    Google Scholar 

  29. Matiz S, Barner KE (2016) Label consistent recursive least squares dictionary learning for image classification. In: IEEE international conference on image processing (ICIP), pp 1888–1892

  30. Milborrow S, Morkel J, Nicolls F (2010) The MUCT landmarked face database. http://www.milbo.org/muct. Accessed 24 Jan 2020

  31. Mitchell T (1999) Cmu face images. https://archive.ics.uci.edu/ml/machine-learning-databases/faces-mld/faces.html. Accessed 7 May 2017

  32. Ou W, You X, Tao D, Zhang P, Tang Y, Zhu Z (2014) Robust face recognition via occlusion dictionary learning. Pattern Recognit 47(4):1559–1572

    Google Scholar 

  33. Patel VM, Wu T, Biswas S, Phillips PJ, Chellappa R (2012) Dictionary-based face recognition under variable lighting and pose. IEEE Trans Inf Forensics Secur 7(3):954–965

    Google Scholar 

  34. Quan Y, Xu Y, Sun Y, Huang Y (2016) Supervised dictionary learning with multiple classifier integration. Pattern Recognit 55:247–260

    MATH  Google Scholar 

  35. Rubinstein R, Peleg T, Elad M (2013) Analysis k-svd: a dictionary-learning algorithm for the analysis sparse model. IEEE Trans Signal Process 61(3):661–677

    MathSciNet  MATH  Google Scholar 

  36. Rubinstein R, Zibulevsky M, Elad M (2008) Efficient implementation of the k-svd algorithm using batch orthogonal matching pursuit. Cs Technion Report CS-2008-08, pp 1–14

  37. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823

  38. Shu T, Zhang B, Tang, YY (2018) Sparse supervised representation-based classifier for uncontrolled and imbalanced classification. IEEE transactions on neural networks and learning systems 1 (Early Access), pp 1–10

  39. Skretting K, Engan K (2010) Recursive least squares dictionary learning algorithm. IEEE Trans Signal Process 58(4):2121–2130

    MathSciNet  MATH  Google Scholar 

  40. Slavkovikj V, Verstockt S, De Neve W, Van Hoecke S, Van de Walle R (2015) Hyperspectral image classification with convolutional neural networks. In: Proceedings of the 23rd ACM international conference on multimedia, ACM, pp 1159–1162

  41. Tang D, Zhu N, Yu F, Chen W, Tang T (2014) A novel sparse representation method based on virtual samples for face recognition. Neural Comput Appl 24(3–4):513–519

    Google Scholar 

  42. Thian NPH, Marcel S, Bengio S (2003) Improving face authentication using virtual samples. In: 2003 IEEE international conference on acoustics, speech, and signal processing, 2003. Proceedings (ICASSP’03), IEEE, pp 3–233

  43. Wang H, Hussain MF, Mukherjee H, Obaidullah SM, Hegadi RS, Roy K, Santosh K (2018) An empirical study: elm in face matching. In: International conference on recent trends in image processing and pattern recognition, Springer, pp 277–287

  44. Wang SJ, Yang J, Sun MF, Peng XJ, Sun MM, Zhou CG (2012) Sparse tensor discriminant color space for face verification. IEEE Trans Neural Netw Learn Syst 23(6):876–888

    Google Scholar 

  45. Wolf L, Hassner T, Maoz I (2011) Face recognition in unconstrained videos with matched backgroundsimilarity. In: IEEE conference on computer vision and pattern recognition, pp 529–534

  46. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Google Scholar 

  47. Xu Y, Li Z, Zhang B, Yang J, You J (2017) Sample diversity, representation effectiveness and robust dictionary learning for face recognition. Inf Sci 375(C):171–182

    Google Scholar 

  48. Xu Y, Zhang B, Zhong Z (2015) Multiple representations and sparse representation for image classification. Pattern Recognit Lett 68:9–14

    Google Scholar 

  49. Xu Y, Zhang Z, Lu G, Yang J (2016) Approximately symmetrical face images for image preprocessing in face recognition and sparse representation based classification. Pattern Recognit 54:68–82

    Google Scholar 

  50. Xu Y, Zhong Z, Yang J, You J, Zhang D (2016) A new discriminative sparse representation method for robust face recognition via l2 regularization. IEEE Trans Neural Netw Learn Syst PP(99):1–10

    Google Scholar 

  51. Xu Y, Zhu Q, Chen Y, Pan JS et al (2012) An improvement to the nearest neighbor classifier and face recognition experiments. Int J Innov Comput Inf Control 8(12):1349–4198

    Google Scholar 

  52. Yong X, Lu Y (2015) Adaptive weighted fusion: a novel fusion approach for image classification. Neurocomputing 168:566–574

    Google Scholar 

  53. Zeng S, Gou J, Deng L (2017) An antinoise sparse representation method for robust face recognition via joint l1 and l2 regularization. Expert Syst Appl 82(1):1–9

    Google Scholar 

  54. Zeng S, Gou J, Yang X (2018) Improving sparsity of coefficients for robust sparse and collaborative representation-based image classification. Neural Comput Appl 30(10):2965–2978

    Google Scholar 

  55. Zeng S, Yang X, Gou J (2017) Multiplication fusion of sparse and collaborative representation for robust face recognition. Multimed Tools Appl 76(20):20889–20907

    Google Scholar 

  56. Zeng S, Yang X, Gou J (2017) Using kernel sparse representation to perform coarse-to-fine recognition of face images. Optik 140:528–535

    Google Scholar 

  57. Zeng S, Zhang B, Du Y (2017) Joint distances by sparse representation and locality-constrained dictionary learning for robust leaf recognition. Comput Electron Agric 142:563–571

    Google Scholar 

  58. Zeng S, Zhang B, Lan Y, Gou J (2019) Robust collaborative representation-based classification via regularization of truncated total least squares. Neural Comput Appl 31(10):5689–5697

    Google Scholar 

  59. Zhang B, Ji S, Li L, Zhang S, Yang W (2016) Sparsity analysis versus sparse representation classifier. Neurocomputing 171:387–393

    Google Scholar 

  60. Zhang B, Karray F, Li Q, Zhang L (2012) Sparse representation classifier for microaneurysm detection and retinal blood vessel extraction. Inf Sci 200:78–90

    Google Scholar 

  61. Zhang B, Vijaya Kumar B, Zhang D (2014) Noninvasive diabetes mellitus detection using facial block color with a sparse representation classifier. IEEE Trans Biomed Eng 61(4):1027–1033

    Google Scholar 

  62. Zhang C, Zhou P, Li C, Liu L (2015) A convolutional neural network for leaves recognition using data augmentation. In: IEEE international conference on computer and information technology; ubiquitous computing and communications; dependable, autonomic and secure computing; pervasive intelligence and computing (CIT/IUCC/DASC/PICOM), Liverpool, pp 2143–2150

  63. Zhang H, Wang F, Chen Y, Zhang W, Wang K, Liu J (2016) Sample pair based sparse representation classification for face recognition. Expert Syst Appl 45:352–358

    Google Scholar 

  64. Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: Which helps face recognition? In: IEEE international conference on computer vision (ICCV), Barcelona, pp 471–478

  65. Zhang Q, Li B (2010) Discriminative k-svd for dictionary learning in face recognition. In: IEEE conference on computer vision and pattern recognition (CVPR), California, pp 2691–2698

  66. Zhang Y, Zeng S, Zeng W, Gou J (2018) Gnn-crc: discriminative collaborative representation-based classification via gabor wavelet transformation and nearest neighbor. J Shanghai Jiaotong Univ (Sci) 23(5):657–665

    Google Scholar 

  67. Zhang Z, Xu Y, Yang J, Li X, Zhang D (2017) A survey of sparse representation: algorithms and applications. IEEE Access 3:490–530

    Google Scholar 

  68. Zhu P, Zhu W, Wang W, Zuo W, Hu Q (2017) Non-convex regularized self-representation for unsupervised feature selection. Image Vis Comput 60:22–29

    Google Scholar 

Download references

Acknowledgements

This research was funded by the University of Macau (no. MYRG2018-00053-FST) and National Natural Science Foundation of China (NSFC) (no. 61976107), Indigenous Innovation’s Capability Development Program of Huizhou University (no. hzu201907), and Science and Technology Program of Huizhou City (Grant no. 2016X0422037). This work was performed in part at the High Performance Computing Cluster (HPCC) which is supported by Information and Communication Technology Office (ICTO) of the University of Macau. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.

Author information

Authors and Affiliations

  1. Pattern Analysis and Machine Intelligence Group, Department of Computer and Information Science, University of Macau, Macau, China

    Shaoning Zeng & Bob Zhang

  2. School of Computer Science and Engineering, Huizhou University, Huizhou, Guangdong, China

    Shaoning Zeng

  3. Electronics and Computer Science, University of Southampton, Southampton, SO17 1BJ, UK

    Yanghao Zhang

  4. College of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang, Jiangsu, China

    Jianping Gou

Authors
  1. Shaoning Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bob Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanghao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianping Gou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bob Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, S., Zhang, B., Zhang, Y. et al. Dual sparse learning via data augmentation for robust facial image classification. Int. J. Mach. Learn. & Cyber. 11, 1717–1734 (2020). https://doi.org/10.1007/s13042-020-01067-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01067-w

Keywords

Search

Navigation