Skip to main content
SpringerLink
Log in

Unlabeled data driven cost-sensitive inverse projection sparse representation-based classification with 1/2 regularization

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Sparse representation-based classification (SRC) has been widely used because it just relies on simple linear regression ideas to do classification, and it does not need learning. However, the performance of SRC is limited by needing sufficient labeled samples per class and the sensitivity to class imbalance. For tackling these problems, an improved SRC model is constructed in this paper. For alleviating the problem of insufficient labeled samples, an unlabeled data-driven inverse projection sparse representation-based classification model is constructed to achieve effective and stable representation and recognition results. The L1/2 and S1/2 regularizations are introduced to capture the sparsity of 1-D and 2-D, and to make the model have good statistical properties. Moreover, the cost-sensitive strategy is integrated into the model’s classification criteria to improve the imbalance of class distribution adaptively, especially for multiclass imbalanced data. A solver utilizing the mixed Gauss-Seidel and Jacobian ADMM algorithm is developed to obtain the approximate solution. Experiments on common public test databases show that the proposed model achieves competitive results compared with the latest published results and some deep-learning algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Yang Q, Wu X D. 10 challenging problems in data mining research. Int J Info Tech Dec Mak, 2006, 05: 597–604

    Article  Google Scholar 

  2. He H B, Garcia E A. Learning from imbalanced data. IEEE Trans Knowl Data Eng, 2009, 21: 1263–1284

    Article  Google Scholar 

  3. Liu X Y, Wu J X, Zhou Z H. Exploratory undersampling for class-imbalance learning. IEEE Trans Syst Man Cybern B, 2009, 39: 539–550

    Article  Google Scholar 

  4. Dudoit S, Fridlyand J. A prediction-based resampling method for estimating the number of clusters in a dataset. Genome Biol, 2002, 3: 1–21

    Article  Google Scholar 

  5. Batista G E A P A, Prati R C, Monard M C. A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explor Newsl, 2004, 6: 20–29

    Article  Google Scholar 

  6. Liu J F, Hu Q H, Yu D. A weighted rough set based method developed for class imbalance learning. Inf Sci, 2008, 178: 1235–1256

    Article  MathSciNet  MATH  Google Scholar 

  7. Wang Y, Hu Q H, Zhou Y C, et al. Local bayes risk minimization based stopping strategy for hierarchical classification. In: Proceedings of IEEE International Conference on Data Mining, New Orleans, 2017. 515–524

    Google Scholar 

  8. Zangeneh V, Shajari M. A cost-sensitive move selection strategy for moving target defense. Comput Secur, 2018, 75: 72–91

    Article  Google Scholar 

  9. Zhao P L, Zhang Y F, Wu M, et al. Adaptive cost-sensitive online classification. IEEE Trans Knowl Data Eng, 2019, 31: 214–228

    Article  Google Scholar 

  10. Khan S H, Hayat M, Bennamoun M, et al. Cost-sensitive learning of deep feature representations from imbalanced data. IEEE Trans Neural Netw Learn Syst, 2018, 29: 3573–3587

    Article  Google Scholar 

  11. Chung Y A, Lin H T, Yang S W. Cost-aware pre-training for multiclass cost-sensitive deep learning. 2015. ArXiv:1511.09337

    Google Scholar 

  12. Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning, Sydney, 2017. 1126–1135

    Google Scholar 

  13. Munkhdalai T, Yu H. Meta networks. In: Proceedings of the 34th International Conference on Machine Learning, 2017. 2554–2563

    Google Scholar 

  14. Schwartz E, Karlinsky L, Shtok J, et al. Delta-encoder: an effective sample synthesis method for few-shot object recognition. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, 2018. 2850–2860

    Google Scholar 

  15. Kwitt R, Hegenbart S, Niethammer M. One-shot learning of scene locations via feature trajectory transfer. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016. 78–86

    Google Scholar 

  16. Wright J, Yang A Y, Ganesh A, et al. Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell, 2009, 31: 210–227

    Article  Google Scholar 

  17. Deng W H, Hu J N, Guo J. Extended SRC: undersampled face recognition via intraclass variant dictionary. IEEE Trans Pattern Anal Mach Intell, 2012, 34: 1864–1870

    Article  Google Scholar 

  18. Du H S, Hu Q P, Qiao D F, et al. Robust face recognition via low-rank sparse representation-based classification. Int J Autom Comput, 2015, 12: 579–587

    Article  Google Scholar 

  19. Zheng C H, Zhang L, Ng T Y, et al. Metasample-based sparse representation for tumor classification. IEEE ACM Trans Comput Biol Bioinf, 2011, 8: 1273–1282

    Article  Google Scholar 

  20. Gan B, Zheng C H, Liu J X. Metasample-based robust sparse representation for tumor classification. ENG, 2013, 05: 78–83

    Article  Google Scholar 

  21. Yang X H, Tian L, Chen Y M, et al. Inverse projection representation and category contribution rate for robust tumor recognition. IEEE ACM Trans Comput Biol Bioinf, 2020, 17: 1262–1275

    Google Scholar 

  22. Yang X H, Wu W M, Chen Y M, et al. An integrated inverse space sparse representation framework for tumor classification. Pattern Recogn, 2019, 93: 293–311

    Article  Google Scholar 

  23. Yang X H, Liu F, Tian L, et al. Pseudo-full-space representation based classification for robust face recognition. Signal Process-Image Commun, 2018, 60: 64–78

    Article  Google Scholar 

  24. Yang X H, Wang Z, Wu H, et al. Stable and compact face recognition via unlabeled data driven sparse representation-based classification. 2021. arXiv:2111.02847

    Google Scholar 

  25. Xu Z B, Zhang H, Wang Y, et al. L 1/2 regularization. Sci China Inf Sci, 2010, 53: 1159–1169

    Article  MathSciNet  MATH  Google Scholar 

  26. Fazel M. Matrix rank minimization with applications. Dissertation for Ph.D. Degree. Stanford: Stanford University, 2002

    Google Scholar 

  27. Rao G, Peng Y, Xu Z B. Robust sparse and low-rank matrix decomposition based on S 1/2 modeling (in Chinese). Sci Sin Inform, 2013, 43: 733–748

    Article  Google Scholar 

  28. Candés E J, Recht B. Exact matrix completion via convex optimization. Found Comput Math, 2009, 9: 717–772

    Article  MathSciNet  MATH  Google Scholar 

  29. Recht B, Fazel M, Parrilo P A. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev, 2010, 52: 471–501

    Article  MathSciNet  MATH  Google Scholar 

  30. Fan J Q, Peng H. Nonconcave penalized likelihood with a diverging number of parameters. Ann Statist, 2004, 32: 928–961

    Article  MathSciNet  MATH  Google Scholar 

  31. Fu W J, Knight K. Asymptotics for lasso-type estimators. Ann Statist, 2000, 28: 1356–1378

    Article  MathSciNet  MATH  Google Scholar 

  32. Xu Z B, Chang X Y, Xu F M, et al. L 1/2 regularization: a thresholding representation theory and a fast solver. IEEE Trans Neural Netw Learn Syst, 2012, 23: 1013–1027

    Article  Google Scholar 

  33. Lu C, Feng J S, Yan S C, et al. A unified alternating direction method of multipliers by majorization minimization. IEEE Trans Pattern Anal Mach Intell, 2018, 40: 527–541

    Article  Google Scholar 

  34. Elkan C. The foundations of cost-sensitive learning. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence, 2001. 973–978

    Google Scholar 

  35. Georghiades A S, Belhumeur P N, Kriegman D J. From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Machine Intell, 2001, 23: 643–660

    Article  Google Scholar 

  36. Sim T, Baker S, Bsat M. The CMU pose, illumination, and expression database. IEEE Trans Pattern Anal Machine Intell, 2003, 25: 1615–1618

    Article  Google Scholar 

  37. Martinez A M, Benavente R. The AR Face Database. CVC Technical Report 24, Purdue University. 1998

    Google Scholar 

  38. Shipp M A, Ross K N, Tamayo P, et al. Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning. Nat Med, 2002, 8: 68–74

    Article  Google Scholar 

  39. Armstrong S A, Staunton J E, Silverman L B, et al. MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nat Genet, 2002, 30: 41–47

    Article  Google Scholar 

  40. Staunton J E, Slonim D K, Coller H A, et al. Chemosensitivity prediction by transcriptional profiling. In: Proceedings of National Academy of Sciences of the United States of America, 2001. 10787–10792

    Google Scholar 

  41. Dudoit S, Fridlyand J, Speed T P. Comparison of discrimination methods for the classification of tumors using gene expression data. J Am Stat Assoc, 2002, 97: 77–87

    Article  MathSciNet  MATH  Google Scholar 

  42. Ahonen T, Hadid A, Pietikainen M. Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell, 2006, 28: 2037–2041

    Article  MATH  Google Scholar 

  43. Tan X Y, Triggs B. Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process, 2010, 19: 1635–1650

    Article  MathSciNet  MATH  Google Scholar 

  44. Chan T H, Jia K, Gao S H, et al. PCANet: a simple deep learning baseline for image classification? IEEE Trans Image Process, 2015, 24: 5017–5032

    Article  MathSciNet  MATH  Google Scholar 

  45. Yang M, Zhang L, Feng X C, et al. Sparse representation based fisher discrimination dictionary learning for image classification. Int J Comput Vis, 2014, 109: 209–232

    Article  MathSciNet  MATH  Google Scholar 

  46. Zhang Z, Li F Z, Chow T W S, et al. Sparse codes auto-extractor for classification: a joint embedding and dictionary learning framework for representation. IEEE Trans Signal Process, 2016, 64: 3790–3805

    Article  MathSciNet  MATH  Google Scholar 

  47. Zhang Z, Jiang W M, Qin J, et al. Jointly learning structured analysis discriminative dictionary and analysis multiclass classifier. IEEE Trans Neural Netw Learn Syst, 2018, 29: 3798–3814

    Article  MathSciNet  Google Scholar 

  48. García V, Salvador Sánchez J. Mapping microarray gene expression data into dissimilarity spaces for tumor classification. Inf Sci, 2015, 294: 362–375

    Article  MathSciNet  Google Scholar 

  49. Gan B, Zheng C H, Zhang J, et al. Sparse representation for tumor classification based on feature extraction using latent low-rank representation. Biomed Res Int, 2014, 2014: 1–7

    Google Scholar 

  50. Yang X H, Jiang X Y, Tian C X, et al. Inverse projection group sparse representation for tumor classification: a low rank variation dictionary approach. Knowl-Based Syst, 2020, 196: 105768

    Article  Google Scholar 

  51. Kolali Khormuji M, Bazrafkan M. A novel sparse coding algorithm for classification of tumors based on gene expression data. Med Biol Eng Comput, 2016, 54: 869–876

    Article  Google Scholar 

  52. Zhang Z, Li F Z, Zhao M B, et al. Joint low-rank and sparse principal feature coding for enhanced robust representation and visual classification. IEEE Trans Image Process, 2016, 25: 2429–2443

    Article  MathSciNet  MATH  Google Scholar 

  53. Li Z M, Zhang Z, Qin J, et al. Discriminative fisher embedding dictionary learning algorithm for object recognition. IEEE Trans Neural Netw Learn Syst, 2020, 31: 786–800

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant No. 41771375), Open Fund of Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education (Grant No. IPIU2019010), and Natural Science Foundations of Henan Province (Grant Nos. 202102310087, 222300420417).

Author information

Authors and Affiliations

  1. Henan Engineering Research Center for Artificial Intelligence Theory and Algorithms, School of Mathematics and Statistics, Henan University, Kaifeng, 475004, China

    Xiaohui Yang & Zheng Wang

  2. National Engineering Laboratory for Big Data Analytics, School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, 750041, China

    Jian Sun & Zongben Xu

Authors
  1. Xiaohui Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zongben Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zongben Xu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Wang, Z., Sun, J. et al. Unlabeled data driven cost-sensitive inverse projection sparse representation-based classification with 1/2 regularization. Sci. China Inf. Sci. 65, 182102 (2022). https://doi.org/10.1007/s11432-021-3319-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-021-3319-4

Keywords

Search

Navigation