Skip to main content

Advertisement

Springer Nature Link
Log in

Partial multi-label learning with noisy side information

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Partial multi-label learning (PML) aims to learn from the training data where each training example is annotated with a candidate label set, among which only a subset is relevant. Despite the success of existing PML approaches, a major drawback of them lies in lacking of robustness to noisy side information. To tackle this problem, we introduce a novel partial multi-label learning with noisy side information approach, which simultaneously removes noisy outliers from the training instances and trains robust partial multi-label classifier for unlabeled instances prediction. Specifically, we first represent the observed sample set as a feature matrix and then decompose it into an ideal feature matrix and an outlier feature matrix by using the low-rank and sparse decomposition scheme, where the former is constrained to be low rank by considering that the noise-free feature information always lies in a low-dimensional subspace and the latter is assumed to be sparse by considering that the outliers are usually sparse among the observed feature matrix. Secondly, we refine an ideal label confidence matrix from the observed label matrix and use the graph Laplacian regularization to constrain the confidence matrix to keep the intrinsic structure among feature vectors. Thirdly, we constrain the feature mapping matrix to be low rank by utilizing the label correlations. Finally, we obtain both the ideal features and ground-truth labels via minimizing the loss function, where the augmented Lagrange multiplier algorithm and quadratic programming are incorporated to solve the optimization problem. Extensive experiments conducted on ten different data sets demonstrate the effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Boutell M, Luo J, Shen X, Brown C (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771

    Article  Google Scholar 

  2. Cabral R, De la Torre F, Costeira JP, Bernardino A (2014) Matrix completion for weakly-supervised multi-label image classification. IEEE Trans Pattern Anal Mach Intell 37(1):121–135

    Article  Google Scholar 

  3. Cai J, Candès E, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982

    Article  MathSciNet  Google Scholar 

  4. Chiang K, Hsieh C, Dhillon I (2015) Matrix completion with noisy side information. In: Advances in neural information processing systems, pp 3447–3455

  5. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(1):1–30

    MathSciNet  MATH  Google Scholar 

  6. Diplaris S, Tsoumakas G, Mitkas P, Vlahavas I (2005) Protein classification with multiple algorithms. In: Panhellenic conference on informatics, pp 448–456

  7. Duygulu P, Barnard K, Freitas J, Forsyth D (2002) Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In: European conference on computer vision, pp 97–112

  8. Fang J, Zhang M (2019) Partial multi-label learning via credible label elicitation. In: AAAI conference on artificial intelligence, pp 3518–3525

  9. Feng L, An B (2018) Leveraging latent label distributions for partial label learning. In: International joint conference on artificial intelligence, pp 2017–2113

  10. Feng L, An B (2019) Partial label learning by semantic difference maximization. In: International joint conference on artificial intelligence, pp 2294–2300

  11. Feng S, Lang C (2018) Graph regularized low-rank feature mapping for multi-label learning with application to image annotation. Multidimens Syst Signal Process 29(4):1351–1372

    Article  Google Scholar 

  12. Goldberg A, Recht B, Xu J, Nowak R, Zhu X (2010) Transduction with matrix completion: three birds with one stone. In: Advances in neural information processing systems, pp 757–765

  13. Guo Y (2017) Convex co-embedding for matrix completion with predictive side information. In: AAAI conference on artificial intelligence, pp 1955–1961

  14. Han Y, Sun G, Shen Y, Zhang X (2018) Multi-label learning with highly incomplete data via collaborative embedding. In: International conference on knowledge discovery and data mining, pp 1494–1503

  15. Huang D, Cabral R, Dela T (2016) Robust regression. IEEE Trans Pattern Anal Mach Intell 38(2):363–375

    Article  Google Scholar 

  16. Huiskes M, Lew M (2008) The mirflickr retrieval evaluation. In: ACM international conference on multimedia information retrieval, pp 39–43

  17. Ji S, Ye J (2009) An accelerated gradient method for trace norm minimization. In: International conference on machine learning, pp 457–464

  18. Katakis I, Tsoumakas G, Vlahavas I (2008) Multilabel text classification for automated tag suggestion. In: Machine learning and knowledge discovery in databases, pp 5–13

  19. Li Z, Lyu G, Feng S (2020) Partial multi-label learning via multi-subspace representation. In: International joint conference on artificial intelligence

  20. Lin Z, Liu R, Su Z (2011) Linearized alternating direction method with adaptive penalty for low-rank representation. In: Advances in neural information processing systems, pp 612–620

  21. Lyu G, Feng S, Li Y (2020) Noisy label tolerance: a new perspective of partial multi-label learning. Inf Sci 543(8):454–466

    MathSciNet  Google Scholar 

  22. Lyu G, Feng S, Li Y (2020) Partial multi-label learning via probabilistic graph matching mechanism. In: ACM SIGKDD conference on knowledge discovery and data mining, pp 105–113

  23. Mencia E, Fürnkranz J (2008) Efficient pairwise multilabel classification for large-scale problems in the legal domain. In: Machine learning and knowledge discovery in databases, pp 50–65

  24. Pestian J, Brew C, Matykiewicz P, Hovermale D, Johnson N, Cohen K, Duch W (2007) A shared task involving multi-label classification of clinical free text. In: Biological, translational, and clinical language processing, pp 97–104

  25. Sun L, Feng S, Wang T, Lang C, Jin Y (2019) Partial multi-label learning by low-rank and sparse decomposition. In: AAAI conference on artificial intelligence, pp 5016–5023

  26. Tsoumakas G, Katakis I, Vlahavas I (2008) Effective and efficient multilabel classification in domains with large number of labels. In: Machine learning and knowledge discovery in databases, pp 53–59

  27. Turnbull D, Barrington L, Torres D, Lanckriet G (2008) Semantic annotation and retrieval of music and sound effects. IEEE Trans Audio Speech Lang Process 16(2):467–476

    Article  Google Scholar 

  28. Wang H, Liu W, Zhao Y, Zhang C, Hu T, Chen G (2019) Discriminative and correlative partial multi-label learning. In: International joint conference on artificial intelligence, pp 3691–3697

  29. Wu J, Wu X, Chen Q, Zhang M (2020) Feature-induced manifold disambiguation for multi-view partial multi-label learning. In: ACM SIGKDD conference on knowledge discovery and data mining, pp 557–565

  30. Xie M, Huang S (2018) Partial multi-label learning. In: AAAI conference on artificial intelligence, pp 4303–4309

  31. Xu M, Jin R, Zhou Z (2013) Speedup matrix completion with side information: application to multi-label learning. In: Advances in neural information processing systems, pp 2301–2309

  32. Xu M, Niu G, Han B, Tsang I, Zhou Z, Sugiyama M (2018) Matrix co-completion for multi-label classification with missing features and labels. arXiv preprint arXiv:1805.09156

  33. Xu N, Liu Y, Geng X (2020) Partial multi-label learning with label distribution. In: AAAI conference on artificial intelligence, pp 6510–6517

  34. Yu G, Chen X, Domeniconi C, Wang J, Li Z, Zhang Z, Wu X (2018) Feature-induced partial multi-label learning. In: International conference on data mining, pp 1398–1403

  35. Yuille A, Rangarajan A (2003) The concave–convex procedure. Neural Comput 15(4):915–936

    Article  Google Scholar 

  36. Zhang M, Wu L (2015) Lift: multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120

    Article  Google Scholar 

  37. Zhang M, Zhou Z (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  38. Zhang Q, Zhong Y, Zhang M (2018) Feature-induced labeling information enrichment for multi-label learning. In: AAAI conference on artificial intelligence, pp. 4446–4453

  39. Zhang X, Ma Y, Lin Z, Gao H, Zhuang L, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. In: IEEE conference on computer vision and pattern recognition, pp. 2328–2335

  40. Zhang Y, Shi D, Gao J, Cheng D (2017) Low-rank-sparse subspace representation for robust regression. In: IEEE conference on computer vision and pattern recognition, pp. 7445–7454

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (No. 61872032), in part by the Beijing Natural Science Foundation (Nos. 4202058, 9192008, 4202057, 4202060), in part by the Fundamental Research Funds for the Central universities (2018YJS038, 2019JBM020) and in part by the Key R&D Program of Zhejiang Province (No. 2019C01068).

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing, 100044, China

    Lijuan Sun, Songhe Feng & Gengyu Lyu

  2. School of Computer and Information Technology, Beijing Jiaotong University, Beijing, 100044, China

    Lijuan Sun, Songhe Feng & Gengyu Lyu

  3. School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China

    Hua Zhang & Guojun Dai

Authors
  1. Lijuan Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Songhe Feng
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Gengyu Lyu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Hua Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Guojun Dai
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Songhe Feng.

Ethics declarations

Conflict of interest

All the authors certify that there is no conflict of interest with any individual/organization for the present work.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, L., Feng, S., Lyu, G. et al. Partial multi-label learning with noisy side information. Knowl Inf Syst 63, 541–564 (2021). https://doi.org/10.1007/s10115-020-01527-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-020-01527-3

Keywords