Skip to main content
SpringerLink
Log in

ML-SLSTSVM: a new structural least square twin support vector machine for multi-label learning

  • Theoretical advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Multi-label learning (MLL) is a special supervised learning task, where any single instance possibly belongs to several classes simultaneously. Nowadays, MLL methods are increasingly required by modern applications, such as protein function classification, speech recognition and textual data classification. In this paper, a structural least square twin support vector machine (SLSTSVM) classifier for multi-label learning is presented. This proposed ML-SLSTSVM focuses on the cluster-based structural information of the corresponding class in each optimization problem, which is vital for designing a good classifier in different real-world problems. This method is extended to a nonlinear version by the kernel trick. Experimental results demonstrate that proposed method is superior in generalization performance to other classifiers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://mulan.sourceforge.net/datasets-mlc.html.

References

  1. Sorower MS (2010) A literature survey on algorithms for multi-label learning, vol 18. Oregon State University, Corvallis

    Google Scholar 

  2. Wu J-S, Huang S-J, Zhou Z-H (2014) Genome-wide protein function prediction through multi-instance multi-label learning. IEEE/ACM Trans Comput Biol Bioinform 11(5):891–902

    Article  Google Scholar 

  3. Wang X, Zhang W, Zhang Q, Li G-Z (2015) MultiP-SChlo: multi-label protein subchloroplast localization prediction with Chou’s pseudo amino acid composition and a novel multi-label classifier. Bioinformatics 31(16):2639–2645

    Article  Google Scholar 

  4. Singh-Miller N, Collins M (2009) Learning label embeddings for nearest-neighbor multi-class classification with an application to speech recognition. In: Advances in neural information processing systems 22 (NIPS 2009), pp 1678–1686

  5. Xu G, Lee H, Koo M-W, Seo J (2017) Convolutional neural network using a threshold predictor for multi-label speech act classification. In: 2017 IEEE international conference on big data and smart computing (BigComp), 2017, pp 126–130

  6. Wu B, Zhong E, Horner A, Yang Q (2014) Music emotion recognition by multi-label multi-layer multi-instance multi-view learning. In: Proceedings of the 22nd ACM international conference on multimedia, 2014, pp 117–126

  7. Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP (2008) Multi-label classification of music into emotions. ISMIR 8:325–330

    Google Scholar 

  8. Liu Y, Wen K, Gao Q, Gao X, Nie F (2018) SVM based multi-label learning with missing labels for image annotation, vol 78. Elsevier Ltd., Amsterdam

    Google Scholar 

  9. Wu J, Yu Y, Huang C, Yu K (2015) Deep multiple instance learning for image classification and auto-annotation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp 3460–3469

  10. Tsoumakas G, Katakis I (2007) Multi-label classification: an overview. Int J Data Warehous Min 3(3):1–13

    Article  Google Scholar 

  11. Zhang M-L, Zhou Z-H (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  12. Fürnkranz J, Hüllermeier E, Loza Mencía E, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153

    Article  Google Scholar 

  13. Elisseeff A, Weston J (2001) A kernel method for multi-labelled classification. In: Advances in neural information processing systems 14 (NIPS 2001), pp 681–687

  14. Xu J (2013) Fast multi-label core vector machine. Pattern Recognit 46(3):885–898

    Article  MATH  Google Scholar 

  15. Chen WJ, Shao YH, Li CN, Deng NY (2016) MLTSVM: a novel twin support vector machine to multi-label learning. Pattern Recognit 52:61–74

    Article  MATH  Google Scholar 

  16. Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048

    Article  MATH  Google Scholar 

  17. Zhang M-L (2009) Ml-rbf: RBF neural networks for multi-label learning. Neural Process Lett 29(2):61–74

    Article  Google Scholar 

  18. Zhang M-L, Zhou Z-H (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351

    Article  Google Scholar 

  19. Clare A, King RD (2001) Knowledge discovery in multi-label phenotype data. In: European conference on principles of data mining and knowledge discovery, 2001, pp 42–53

    Chapter  MATH  Google Scholar 

  20. Vens C, Struyf J, Schietgat L, Džeroski S, Blockeel H (2008) Decision trees for hierarchical multi-label classification. Mach Learn 73(2):185–214

    Article  Google Scholar 

  21. Qi Z, Tian Y, Shi Y (2013) Structural twin support vector machine for classification. Knowl Based Syst 43:74–81

    Article  Google Scholar 

  22. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Article  Google Scholar 

  23. Arun Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Article  Google Scholar 

  24. Hüllermeier E, Fürnkranz J, Cheng W, Brinker K (2008) Label ranking by learning pairwise preferences. Artif Intell 172(16–17):1897–1916

    Article  MathSciNet  MATH  Google Scholar 

  25. Sun X et al (2016) ELM-ML: study on multi-label classification using extreme learning machine. In: Proceedings of ELM-2015, vol 2. Springer, 2016, pp 107–116

  26. Schapire RE, Singer Y (2000) BoosTexter: a boosting-based system for text categorization. Mach Learn 39(2–3):135–168

    Article  MATH  Google Scholar 

  27. Wang Y et al (2017) A multi-label learning method for efficient affective detection. In: 2017 IEEE EMBS international conference on biomedical and health informatics, pp 61–64

  28. Reyes O, Morell C, Ventura S (2018) Effective active learning strategy for multi-label learning. Neurocomputing 273:494–508

    Article  Google Scholar 

  29. Li Z, Tang J, Mei T (2018) Deep collaborative embedding for social image understanding. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2018.2852750

    Article  Google Scholar 

  30. Zhu J, Liao S, Lei Z, Li SZ (2017) Multi-label convolutional neural network based pedestrian attribute classification. Image Vis Comput 58:224–229

    Article  Google Scholar 

  31. Zhuang N, Yan Y, Chen S, Wang H, Shen C (2018) Multi-label learning based deep transfer neural network for facial attribute classification. Pattern Recognit 80:225–240

    Article  Google Scholar 

  32. Li Z, Tang J (2017) Weakly supervised deep matrix factorization for social image understanding. IEEE Trans Image Process 26(1):276–288

    Article  MathSciNet  MATH  Google Scholar 

  33. Cakir E, Heittola T, Huttunen H, Virtanen T (2015) Multi-label vs. combined single-label sound event detection with deep neural networks. In: 2015 23rd European signal processing conference (EUSIPCO), 2015, pp 2551–2555

  34. Barutcuoglu Z, Schapire RE, Troyanskaya OG (2006) Hierarchical multi-label prediction of gene function. Bioinformatics 22(7):830–836

    Article  Google Scholar 

  35. Zhang W, Yan J, Wang X, Zha H (2018) Deep extreme multi-label learning. In: Proceedings of the 2018 ACM on international conference on multimedia retrieval. ACM, pp 100–107

  36. Prabhu Y, Varma M (2014) Fastxml: a fast, accurate and stable tree-classifier for extreme multi-label learning. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, 2014, pp 263–272

  37. Weston J, Makadia A, Yee H (2013) Label partitioning for sublinear ranking. In: International conference on machine learning, 2013, pp 181–189

  38. Xu C, Tao D, Xu C (2016) Robust extreme multi-label learning. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp 1275–1284

  39. Bhatia K, Jain H, Kar P, Varma M, Jain P (2015) Sparse local embeddings for extreme multi-label classification. In: Advances in neural information processing systems 28 (NIPS 2015), pp 730–738

  40. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  MATH  Google Scholar 

  41. Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999

    Article  Google Scholar 

  42. Hofmann T, Schölkopf B, Smola AJ (2008) Kernel methods in machine learning. Ann Stat 36(3):1171–1220

    Article  MathSciNet  MATH  Google Scholar 

  43. Auria L, Moro R (2008) Support vector machines (SVM) as a technique for solvency analysis. DIW Berlin discussion paper no. 811. Available at SSRN: https://ssrn.com/abstract=1424949 or https://doi.org/10.2139/ssrn.1424949

  44. Xue H, Chen S, Yang Q (2011) Structural regularized support vector machine: a framework for structural large margin classifier. IEEE Trans Neural Netw 22(4):573–587

    Article  Google Scholar 

  45. Shao Y-H, Zhang C-H, Wang X-B, Deng N-Y (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968

    Article  Google Scholar 

  46. Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceeding of ACM SIGKDD international conference on knowledge discovery and data mining—KDD’01, pp 77–86

  47. Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12(Jul):2411–2414

    MathSciNet  MATH  Google Scholar 

  48. Nasiri JA, Charkari NM, Jalili S (2015) Least squares twin multi-class classification support vector machine. Pattern Recognit 48(3):984–992

    Article  MATH  Google Scholar 

  49. Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics Bull 1(6):80–83

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, University of Zanjan, Zanjan, Iran

    Meisam Azad-Manjiri & Ali Amiri

  2. Department of Computer Engineering, Iran University of Science and Technology, Tehran, Iran

    Alireza Saleh Sedghpour

Authors
  1. Meisam Azad-Manjiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Amiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alireza Saleh Sedghpour
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meisam Azad-Manjiri.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Azad-Manjiri, M., Amiri, A. & Saleh Sedghpour, A. ML-SLSTSVM: a new structural least square twin support vector machine for multi-label learning. Pattern Anal Applic 23, 295–308 (2020). https://doi.org/10.1007/s10044-019-00779-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-019-00779-2

Keywords

Search

Navigation