Skip to main content
SpringerLink
Log in

An improved MLTSVM using label-specific features with missing labels

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-label twin support vector machine (MLTSVM) is an excellent multi-label classification algorithm, which has attracted much attention. Although MLTSVM can effectively solve the multi-label classification problem, it has some drawbacks. a) MLTSVM uses the same feature representation for each label, but in practice, each label has its own specific features. Therefore, MLTSVM might not obtain the optimal classification results. b) In practical applications, there are a large number of samples with missing labels and only a small number of samples with complete labels, because it is expensive to obtain all labels of samples. However, MLTSVM can only use expensive samples with complete labels, not cheap samples with missing labels. For the above drawbacks, we propose an improved MLTSVM using label-specific features with missing labels (LSFML-MLTSVM) in this paper. LSFML-MLTSVM first extracts label-specific features via using semi-supervised clustering analysis and then obtains the structure information of samples and the geometry information of the marginal distribution. Furthermore, in the label-specific feature space, the above two valuable information is introduced into MLTSVM to reconstruct the classification model. Finally, the successive overrelaxation (SOR) algorithm is used to solve the classification model efficiently. Experimental results on benchmark multi-label datasets show that LSFML-MLTSVM has better classification performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. The multi-label benchmark datasets are available in the multi-label classification dataset repository, http://www.uco.es/kdis/mllresources/#HallEtAl2009.

References

  1. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    Article  MATH  Google Scholar 

  2. Doroudyan MH, Niaki STA (2021) Pattern recognition in financial surveillance with the ARMA-GARCH time series model using support vector machine. Expert Syst Appl 182:115334

    Article  Google Scholar 

  3. Islam MMM, Prosvirin AE, Kim JM (2021) Data-driven prognostic scheme for rolling-element bearings using a new health index and variants of least-square support vector machines. Mech Syst Signal Process 160:107853

    Article  Google Scholar 

  4. Li X, Yang J, Fan Y, Xie M, Qian X, Li H (2021) Rapid monitoring of heavy metal pollution in lake water using nitrogen and phosphorus nutrients and physicochemical indicators by support vector machine. Chemosphere 280:130599

    Article  Google Scholar 

  5. Sanchez Lasheras JE, Sanchez Lasheras F, Gonzalez Donquiles C, Tardon A, Castano-Vinyals G, Palazuelos C, Salas D, Martin Sanchez V, de Cos Juez FJ (2021) Hybrid algorithm for the classification of prostate cancer patients of the mcc-Spain study based on support vector machines and genetic algorithms. Neurocomputing 452:386–394

    Article  Google Scholar 

  6. Kurilova V, Goga J, Oravec M, Pavlovicova J, Kajan S (2021) Support vector machine and deep-learning object detection for localisation of hard exudates. Sci Rep 11(1):16045

    Article  Google Scholar 

  7. Deng N, Tian Y, Zhang C (2012) Support vector machines: optimization based theory, algorithms, and extensions. Chapman & Hall/CRC, New York

    Book  MATH  Google Scholar 

  8. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  MATH  Google Scholar 

  9. Che Z, Liu B, Xiao Y, Cai H (2021) Twin support vector machines with privileged information. Inf Sci 573:141–153

    Article  MathSciNet  Google Scholar 

  10. Prasad SC, Balasundaram S (2021) On lagrangian L2-norm pinball twin bounded support vector machine via unconstrained convex minimization. Inf Sci 571:279–302

    Article  MathSciNet  Google Scholar 

  11. Sharma S, Rastogi R, Chandra S (2021) Large-scale twin parametric support vector machine using pinball loss function. IEEE Trans Syst Man Cybern Syst 51(2):987–1003

    Article  Google Scholar 

  12. Richhariya B, Tanveer M (2021) A fuzzy universum least squares twin support vector machine (FULSTSVM). Neural Comput Applic

  13. Ma J, Yang L, Sun Q (2021) Adaptive robust learning framework for twin support vector machine classification. Knowl-Based Syst 211:106536

    Article  Google Scholar 

  14. Xie J, Hone K, Xie W, Gao X, Shi Y, Liu X (2013) Extending twin support vector machine classifier for multi-category classification problems. Intell Data Aanal 17(4):649–664

    Article  Google Scholar 

  15. Tomar D, Agarwal S (2015) A comparison on multi-class classification methods based on least squares twin support vector machine. Knowl-Based Syst 81:131–147

    Article  Google Scholar 

  16. Gao Z, Fang S, Gao X, Luo J, Medhin N (2021) A novel kernel-free least squares twin support vector machine for fast and accurate multi-class classification. Knowl-Based Syst 226:107123

    Article  Google Scholar 

  17. Yang Z, Shao Y, Zhang X (2013) Multiple birth support vector machine for multi-class classification. Neural Comput Applic 22(Supplement-1):153–161

    Article  Google Scholar 

  18. Ding S, Sun Y, An Y, Jia W (2020) Multiple birth support vector machine based on recurrent neural networks. Appl Intell 50(7):2280–2292

    Article  Google Scholar 

  19. Wen Y, Ma J, Yuan C, Yang L (2020) Projection multi-birth support vector machinea for multi-classification. Appl Intell 50(10):3040–3056

    Article  Google Scholar 

  20. Ding S, Zhang J, Zhang X, An Y (2018) Survey on multi class twin support vector machines. Ruan Jian Xue Bao/J Sofw 29(1):89–108

    MathSciNet  MATH  Google Scholar 

  21. Tomar D, Agarwal S (2015) A comparison on multi-class classification methods based on least squares twin support vector machine. Knowl-Based Syst 81:131–147

    Article  Google Scholar 

  22. Gu H, Niu B, Gao Z (2014) A directed acyclic graph algorithm for multi-class classification based on twin support vector machine. J Inf Comput Sci 11(18):6529–6536

    Article  Google Scholar 

  23. Xu Y, Guo R, Wang L (2013) A twin multi-class classification support vector machine. Cogn Comput 5(4):580–588

    Article  Google Scholar 

  24. de Lima MD, Costa NL, Barbosa R (2018) Improvements on least squares twin multi-class classification support vector machine. Neurocomputing 313:196–205

    Article  Google Scholar 

  25. Ai Q, Wang A, Wang Y, Sun H (2019) An improved Twin-KSVC with its applications. Neural Comput Applic 31(10):6615– 6624

    Article  Google Scholar 

  26. Qiang W, Zhang J, Zhen L, Jing L (2020) Robust weighted linear loss twin multi-class support vector regression for large-scale classification. Signal Process 170:107449

    Article  Google Scholar 

  27. Bendjoudi I, Vanderhaegen F, Hamad D, Dornaika F (2021) Multi-label, multi-task CNN approach for context-based emotion recognition. Inf Fusion 76:422–428

    Article  Google Scholar 

  28. Singh S, Majumdar A (2020) Non-intrusive load monitoring via multi-label sparse representation-based classification. IEEE Trans Smart Grid 11(2):1799–1801

    Article  Google Scholar 

  29. Sun Z, Wang C, Zhao Y, Yan C (2020) Multi-label ECG signal classification based on ensemble classifier. IEEE Access 8:117986–117996

    Article  Google Scholar 

  30. Ji Z, Cui B, Zhao Y, Li H, Jiang Y, Xiang T, Hospedales T, Fu Y (2020) Deep ranking for image zero-shot multi-label classification. IEEE Trans Image Process 29:6549–6560

    Article  MathSciNet  MATH  Google Scholar 

  31. Qin Q, Huang L, Wei Z (2020) Deep multilevel similarity hashing with fine-grained features for multi-label image retrieval. Neurocomputing 409:46–59

    Article  Google Scholar 

  32. Bogaert M, Lootens J, Van den Poel D, Ballings M (2019) Evaluating multi-label classifiers and recommender systems in the financial service sector. Eur J Oper Res 279(2):620–634

    Article  Google Scholar 

  33. Zhang M, Zhou Z (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  34. Wu G, Tian Y, Zhang C (2018) A unified framework implementing linear binary relevance for multi-label learning. Neurocomputing 289:86–100

    Article  Google Scholar 

  35. Xie J, Yu L, Zhu L, Duan G (2019) Conditional entropy based classifier chains for multi-label classification. Neurocomputing 335:185–194

    Article  Google Scholar 

  36. Liu B, Tsoumakas G (2020) Dealing with class imbalance in classifier chains via random undersampling. Knowl-Based Syst 192:105292

    Article  Google Scholar 

  37. Rastin N, Taheri M, Jahromi MZ (2021) A stacking weighted k-nearest neighbour with thresholding. Inf Sci 571:605–622

    Article  MathSciNet  Google Scholar 

  38. Zhang M, Zhou Z (2007) ML-KNN: A lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048

    Article  MATH  Google Scholar 

  39. Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification Advances in Neural Information Processing Systems, vol 14. MIT Press, Cambridge, pp 681–687

  40. Wu G, Zheng R, Tian Y, Liu D (2020) Joint ranking svm and binary relevance with robust low-rank learning for multi-label classification. Neural Netw 122:24–39

    Article  MATH  Google Scholar 

  41. Wang H, Xu Y (2021) Sparse elastic net multi-label rank support vector machine with pinball loss and its applications. Appl Soft Comput 104:107232

    Article  Google Scholar 

  42. Chen W, Shao Y, Li C, Deng N (2016) MLTSVM: A novel twin support vector machine to multi-label learning. Pattern Recogn 52:61–74

    Article  MATH  Google Scholar 

  43. Ai Q, Kang Y, Wang A (2021) A novel semi-supervised multi-label twin support vector machine. Intell Autom Soft Comput 27(1):205–220

    Article  Google Scholar 

  44. Ai Q, Kang Y, Wang A, Li X, Li F (2020) An effective semi-supervised multi-label least squares twin support vector machine. IEEE Access 8:213460–213472

    Article  Google Scholar 

  45. Zhang M, Wu L (2015) LIFT: multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120

    Article  MathSciNet  Google Scholar 

  46. Weng W, Lin Y, Wu S, Li Y, Kang Y (2018) Multi-label learning based on label-specific features and local pairwise label correlation. Neurocomputing 273:385–394

    Article  Google Scholar 

  47. Huang J, Li G, Huang Q, Wu X (2018) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern 48(3):876–889

    Article  Google Scholar 

  48. Ma J, Zhang H, Chow TWS (2021) Multilabel classification with label-specific features and classifiers: A coarse- and fine-tuned framework. IEEE Trans Cybern 51(2):1028–1042

    Article  Google Scholar 

  49. Chen Z, Zhang M (2019) Multi-label learning with regularization enriched label-specific features Proceedings of The Eleventh Asian Conference on Machine Learning, vol 101. PMLR, Nagoya, pp 411–424

  50. Guo Y, Chung F, Li G, Wang J, Gee JC (2019) Leveraging label-specific discriminant mapping features for multi-label learning. ACM Trans Knowl Discov Data 13(2):24

    Article  Google Scholar 

  51. Wang Y, Zheng W, Cheng Y, Zhao D (2020) Joint label completion and label-specific features for multi-label learning algorithm. Soft Comput 24(9):6553–6569

    Article  Google Scholar 

  52. Yu Z, Zhang M (2021) Multi-label classification with label-specific feature generation: a wrapped approach. IEEE Trans Pattern Anal Mach Intell

  53. Chen Y, Weng W, Wu S, Chen B, Fan Y, Liu J (2021) An efficient stacking model with label selection for multi-label classification. Appl Intell 51(1):308–325

    Article  Google Scholar 

  54. Huang J, Li G, Huang Q, Wu X (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323

    Article  Google Scholar 

  55. Zhang J, Li C, Cao D, Lin Y, Su S, Dai L, Li S (2018) Multi-label learning with label-specific features by resolving label correlations. Knowl-Based Syst 159:148–157

    Article  Google Scholar 

  56. Weng W, Chen Y, Chen C, Wu S, Liu J (2020) Non-sparse label specific features selection for multi-label classification. Neurocomputing 377:85–94

    Article  Google Scholar 

  57. Jia X, Zhu S, Li W (2020) Joint label-specific features and correlation information for multi-label learning. J Comput Sci Technol 35(2):247–258

    Article  Google Scholar 

  58. Huang J, Qin F, Zheng X, Cheng Z, Yuan Z, Zhang W, Huang Q (2019) Improving multi-label classification with missing labels by learning label-specific features. Inf Sci 492:124–146

    Article  MathSciNet  MATH  Google Scholar 

  59. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(36):2399–2434

    MathSciNet  MATH  Google Scholar 

  60. Mangasarian OL, Musicant DR (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Netw 10(5):1032–1037

    Article  Google Scholar 

  61. Chang C.-C., Lin C.-J. (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27

    Article  Google Scholar 

  62. Demšar J. (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

We thank all anonymous reviewers for their helpful comments, which improved the quality of this paper.

Funding

This work was supported in part by the Natural Science Foundation of Liaoning province in China (2020-MS-281) and Talent Cultivation Project of University of Science and Technology Liaoning in China (2018RC05).

Author information

Authors and Affiliations

  1. School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan, 114051, China

    Qing Ai, Fei Li, Ji Zhao, Qingyun Gao & Fei Zhao

  2. State Grid Information and Telecommunication Group Company Ltd., State Grid Corporation of China, Beijing, 100053, China

    Xiangna Li

  3. College of Information Science and Engineering, Northeastern University, Shenyang, 110819, China

    Wenhui Wang

Authors
  1. Qing Ai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangna Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ji Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wenhui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qingyun Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Fei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qing Ai.

Ethics declarations

Conflict of interest

The author declares that there are no conflicts of interest in the publication of this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ai, Q., Li, F., Li, X. et al. An improved MLTSVM using label-specific features with missing labels. Appl Intell 53, 8039–8060 (2023). https://doi.org/10.1007/s10489-022-03634-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03634-w

Keywords

Search

Navigation