Skip to main content
SpringerLink
Log in

An active multi-class classification using privileged information and belief function

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Many classification models, based on support vector machine, have been designed so far to improve classification performance in both supervised and semi-supervised learning. One of the studies which is done in this case is about the use of privileged information that is hidden in training data. However, the challenge is how to find the privileged information. In most researches, experts have defined privileged information, but in this paper, it has been tried to automatically select a feature as privileged information and classify training data into several groups. This grouping has been used to correct the decision function of classifier. Moreover, the proposed classifier has been used in one-against-all (OAA) approach for semi-supervised datasets. To overcome uncertain areas in OAA, belief function and active learning techniques are applied to extract the most informative samples. The experimental results indicate the superiority of the proposed method among the other state-of-the-art methods in terms of classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Han J, Pei J, Kamber M (2011) Data mining: concepts and techniques. Elsevier, Amsterdam

    MATH  Google Scholar 

  2. Vapnik V, Vashist A (2009) A new learning paradigm: learning using privileged information. Neural Netw 22(5–6):544–557

    Article  Google Scholar 

  3. Bendtsen C, Degasperi A, Ahlberg E, Carlsson L (2017) Improving machine learning in early drug discovery. Ann Math Artif Intell 81(1–2):155–166

    Article  MathSciNet  Google Scholar 

  4. Vrigkas M, Nikou C, Kakadiaris IA (2016) Active privileged learning of human activities from weakly labeled samples. In: Image processing (ICIP), pp 3036–3040

  5. Yan Y, Nie F, Li W, Gao C, Yang Y, Xu D (2016) Image classification by cross-media active learning with privileged information. IEEE Trans Multimed 18(12):2494–2502

    Article  Google Scholar 

  6. Serra-Toro C, Traver VJ, Pla F (2014) Exploring some practical issues of SVM+: is really privileged information that helps? Pattern Recognit Lett 42:40–46

    Article  Google Scholar 

  7. Chang CC, Chien LJ, Lee YJ (2011) A novel framework for multi-class classification via ternary smooth support vector machine. Pattern Recognit 44(6):1235–1244

    Article  Google Scholar 

  8. Bourke C, Deng K, Scott SD et al (2008) On reoptimizing multi-class classifiers. Mach Learn 71(2–3):219–242. https://doi.org/10.1007/s10994-008-5056-8

    Article  Google Scholar 

  9. Lin HY (2012) Efficient classifiers for multi-class classification problems. Decis Support Syst 53(3):473–481

    Article  Google Scholar 

  10. Wu TF, Lin CJ, Weng RC (2004) Probability estimates for multi-class classification by pairwise coupling. J Mach Learn Res 5:975–1005

    MathSciNet  MATH  Google Scholar 

  11. Li CN, Huang YF, Wu HJ, Shao YH, Yang ZM (2016) Multiple recursive projection twin support vector machine for multi-class classification. Int J Mach Learn Cybern 7(5):729–740

    Article  Google Scholar 

  12. Liu B, Xiao Y, Cao L (2017) SVM-based multi-state-mapping approach for multi-class classification. Knowl Based Syst 129:79–96

    Article  Google Scholar 

  13. Tomar D, Agarwal S (2015) A comparison on multi-class classification methods based on least squares twin support vector machine. Knowl Based Syst 81:131–147

    Article  Google Scholar 

  14. Yang ZM, Wu HJ, Li CN, Shao YH (2016) Least squares recursive projection twin support vector machine for multi-class classification. Int J Mach Learn Cybern 7(3):411–426

    Article  Google Scholar 

  15. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  16. Niu L, Shi Y, Wu J (2012) Learning using privileged information with L-1 support vector machine. In: Proceedings of the IEEE/WIC/ACM international joint conferences on web intelligence and intelligent agent technology-volume. IEEE Computer Society, pp 10–14

  17. Niu L, Shi Y, Wu J (2012) Nonlinear l-1 support vector machines for learning using privileged information. In: IEEE 12th international conference on data mining workshops (ICDMW), pp 495–499

  18. Lapin M, Hein M, Schiele B (2014) Learning using privileged information: SVM+ and weighted SVM. Neural Netw 53:95–108

    Article  Google Scholar 

  19. Qi Z, Tian Y, Niu L, Wang B (2015) Semi-supervised classification with privileged information. Int J Mach Learn Cybernet 6(4):667–676

    Article  Google Scholar 

  20. Meng F, Qi Z, Tian Y, Niu L (2018) Pedestrian detection based on the privileged information. Neural Comput Appl 29(12):1485–1494

    Article  Google Scholar 

  21. Liu J, Zhu W, Zhong P (2013) A new multi-class support vector algorithm based on privileged information. J Inf Comput Sci 10(2):443–450

    Google Scholar 

  22. Liu ZG, Pan Q, Dezert J (2013) A new belief-based K-nearest neighbor classification method. Pattern Recognit 46(3):834–844

    Article  Google Scholar 

  23. Ji Y, Sun S, Lu Y (2012) Multitask multiclass privileged information support vector machines. In: Pattern recognition (ICPR), pp 2323–2326

  24. Hamidzadeh J, Sadeghi R, Namaei N (2017) Weighted support vector data description based on chaotic bat algorithm. Appl Soft Comput 60:540–551

    Article  Google Scholar 

  25. Sadeghi R, Hamidzadeh J (2018) Automatic support vector data description. Soft Comput 22(1):147–158

    Article  Google Scholar 

  26. Zhang W (2015) Support vector data description using privileged information. Electron Lett 51(14):1075–1076

    Article  Google Scholar 

  27. Zhu W, Zhong P (2014) A new one-class SVM based on hidden information. Knowl Based Syst 60:35–43

    Article  Google Scholar 

  28. Sharmanska V, Quadrianto N, Lampert CH (2013) Learning to rank using privileged information. In: Computer vision (ICCV), pp 825–832

  29. Wang R, Chow CY, Kwong S (2016) Ambiguity-based multiclass active learning. IEEE Trans Fuzzy Syst 24(1):242–248

    Article  Google Scholar 

  30. Wang S, Tao D, Yang J (2016) Relative attribute SVM+ learning for age estimation. IEEE Trans Cybern 46(3):827–839

    Article  Google Scholar 

  31. Fouad S, Tino P, Raychaudhury S, Schneider P (2013) Incorporating privileged information through metric learning. IEEE Trans Neural Netw Learn Syst 24(7):1086–1098

    Article  Google Scholar 

  32. Li X, Du B, Zhang Y, Xu C, Tao D (2019) Iterative privileged learning. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2018.2889906

    Article  Google Scholar 

  33. Tong S, Koller D (2001) Support vector machine active learning with applications to text classification. J Mach Learn Res 2:45–66

    MATH  Google Scholar 

  34. Dasgupta S (2011) Two faces of active learning. Theor Comput Sci 412(19):1767–1781

    Article  MathSciNet  Google Scholar 

  35. Cai W, Zhang M, Zhang Y (2017) Batch mode active learning for regression with expected model change. IEEE Trans Neural Netw Learn Syst 28(7):1668–1681

    Article  MathSciNet  Google Scholar 

  36. Bouguelia MR, Nowaczyk S, Santosh KC, Verikas A (2018) Agreeing to disagree: active learning with noisy labels without crowdsourcing. Int J Mach Learn Cybern 9(8):1307–1319

    Article  Google Scholar 

  37. Guo H, Wang W (2015) An active learning-based SVM multi-class classification model. Pattern Recognit 48(5):1577–1597

    Article  Google Scholar 

  38. Denux Denoeux T, Smets P (2006) Classification using belief functions: relationship between case-based and model-based approaches. IEEE Trans Syst Man Cybern Part B (Cybern) 36(6):1395–1406

    Article  Google Scholar 

  39. Li F, Qian Y, Wang J, Liang J (2017) Multigranulation information fusion: a Dempster–Shafer evidence theory-based clustering ensemble method. Inf Sci 378:389–409

    Article  Google Scholar 

  40. Panda M, Mishra D, Mishra S (2018) Ensemble methods for improving classifier performance. In: International proceedings on advances in soft computing, intelligent systems and applications. Springer, pp 363–374

  41. Yager RR, Alajlan N (2015) Dempster–Shafer belief structures for decision making under uncertainty. Knowl Based Syst 80:58–66

    Article  Google Scholar 

  42. Moslemnejad S, Hamidzadeh J (2019) A hybrid method for increasing the speed of SVM training using belief function theory and boundary region. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-019-00944-3

    Article  Google Scholar 

  43. Ghosh R, Kumar P, Roy PP (2018) A Dempster-Shafer theory based classifier combination for online signature recognition and verification systems. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-018-0883-9

    Article  Google Scholar 

  44. Liu ZG, Pan Q, Dezert J, Mercier G (2014) Credal classification rule for uncertain data based on belief functions. Pattern Recognit 47(7):2532–2541

    Article  Google Scholar 

  45. Hooshmand Moghaddam V, Hamidzadeh J (2016) New hermite orthogonal polynomial kernel and combined kernels in support vector machine classifier. Pattern Recognit 60:921–935

    Article  Google Scholar 

  46. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  47. Zhu X, Wu X (2004) Class noise vs. attribute noise: a quantitative study. Artif Intell Rev 22(3):177–210

    Article  Google Scholar 

  48. Vajda S, Santosh KC (2017) A fast k-nearest neighbor classifier using unsupervised clustering. In: International conference on recent trends in image processing and pattern recognition, pp 185–193

    Google Scholar 

Download references

Acknowledgements

The authors are grateful to the suggestions of the anonymous reviewers and editor which greatly improved the paper.

Author information

Authors and Affiliations

  1. Faculty of Computer Engineering and Information Technology, Sadjad University of Technology, Mashhad, Iran

    Mitra Javid & Javad Hamidzadeh

Authors
  1. Mitra Javid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Javad Hamidzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Javad Hamidzadeh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Javid, M., Hamidzadeh, J. An active multi-class classification using privileged information and belief function. Int. J. Mach. Learn. & Cyber. 11, 511–524 (2020). https://doi.org/10.1007/s13042-019-00991-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-019-00991-w

Keywords

Search

Navigation