Skip to main content
SpringerLink
Log in

Alignment Based Kernel Selection for Multi-Label Learning

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Kernel based methods are increasingly being used for data modeling because of their conceptual simplicity and outstanding performance on many learning tasks. And kernel alignment, which is usually employed to select particular kernel for a learning task, is an effective quantity to measure the degree of agreement between a kernel and a learning task. However, the existing kernel alignment methods are usually developed for single-label classification problems. In this paper, we consider kernel alignment for multi-label learning to address the problem of kernel selection. Our basic idea is that, firstly an ideal kernel is presented in terms of multiple labels. Then kernel is selected by selecting the parameters of a linear combination of base kernels through maximizing the alignment value between the combined kernel and ideal kernel, and the selected kernel is employed in the binary relevance approach for multi-label learning to construct SVMs as classifiers. Furthermore, our proposed method is improved by considering local kernel alignment criterion. Our idea of selecting kernels by kernel alignment for multi-label learning is experimentally demonstrated to be effective in terms of classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Amayri O, Bouguila N (2010) A study of spam filtering using support vector machines. Artif Intell Rev 34(1):73–108

    Article  Google Scholar 

  2. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, New York

    Book  MATH  Google Scholar 

  3. Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  4. Cristianini N, Elisseeff A, Shawe-Taylor J, Kandola J (2001) On kernel-target alignment. In: Neural information processing systems 14 (NIPS 14)

  5. Cristianini N, Kandola J, Elisseeff A, Shawe-Taylor J (2006) On kerneltarget alignment. Innov Mach Learn 205–256

  6. Wang T, Zhao D, Tian S (2015) An overview of kernel alignment and its applications. Artif Intell Rev 43:179–192

    Article  Google Scholar 

  7. Cortes C, Mohri M, Rostamizadeh A (2012) Algorithms for learning kernels based on centered alignment. J Mach Learn Res 13:795–828

    MathSciNet  MATH  Google Scholar 

  8. Kandola J, Shawe-Taylor J, Cristianini N (2002a) On the extensions of kernel alignment. Technical report 120, Department of Computer Science, University of London, London

  9. Kandola J, Shawe-Taylor J, Cristianini N (2002b) Optimizing kernel alignment over combinations of kernels. Technical report 121, Department of Computer Science, University of London, London

  10. Igel C, Glasmachers T, Mersch B, Pfeifer N, Meinicke P (2007) Gradient-based optimization of kernel-target alignment for sequence kernels applied to bacterial gene start detection. IEEE/ACM Trans Comput Biol 4(2):1–11

    Article  Google Scholar 

  11. Wong William WL, Burkowski FJ (2011) Using kernel alignment to select features of molecular descriptors in a QSAR study. IEEE/ACM Trans Comput Biol 8(5):1373–1384

    Article  Google Scholar 

  12. Liu X-W, Yin J-P, Wang L, Liu L-Q, Liu J, Hou C-P, Zhang J (2013) An adaptive approach to learning optimal neighborhood kernels. IEEE T Cybernet 43(1):371–384

    Article  Google Scholar 

  13. Liu X-W, Wang L, Yin J-P, Zhu E, Zhang J (2013) An efficient approach to integrating radius information into multiple kernel learning. IEEE T Cybern 43(2):557–569

    Article  Google Scholar 

  14. Li M-M, Liu X-W, Wang L, Dou Y, Yin J-P, Zhu E (2016) Multiple kernel clustering with local kernel alignment maximization. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, pp 1704–1710

  15. Camargo JE, González FA (2009) A multi-class kernel alignment method for image collection summarization. In: Conference on progress in pattern recognition. Springer, New York, pp 545–552

  16. Kawanabe M, Nakajima S, Binder A (2009) A procedure of adaptive kernel combination with kernel-target alignment for object classification. In: International conference on image & video retrieval. ACM, No. 23

  17. Yanting L, Wang L, Jianfeng L, JingyuYang C (2014) Multiple kernel clustering based on centered kernel alignment. Pattern Recognit 47:3656–3664

    Article  MATH  Google Scholar 

  18. Wang T, Zhao D, Tian S (2015) An overview of kernel alignment and its applications. Artif Intell Rev 43(2):179–192

    Article  Google Scholar 

  19. Cortes C, Mohri M, Rostamizadeh A (2010) Two-stage learning kernel algorithms. In: International conference on machine learning, pp 239–246

  20. Schapire RE, Singer Y (1999) Improved boosting algorithms using confidence-rated prediction. Mach Learn 37(3):297–336

    Article  MATH  Google Scholar 

  21. Zhang M-L, Zhou Z-H (2007) ML-kNN: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048

    Article  MATH  Google Scholar 

  22. Clare A, King RD (2001) Knowledge discovery in multi-label phenotype data. In: Proceedings of the 5th European conference on principles and practice of knowledge discovery in databases. Springer, Freiburg, Germany, pp 42–53

  23. Zhang M-L, Zhou Z-H (2006) Multi-label neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351

    Article  Google Scholar 

  24. Zhang M-L, Wang Z-J (2009) MIMLRBF: RBF neural networks for multi-instance multi-label learning. Neurocomputing 72(16):3951–3956

    Article  Google Scholar 

  25. Bi W, Kwok J (2013) Efficient multi-label classification with many labels. In: Proceedings of the 30th international conference on machine learning. Atlanta, GA, pp 405–413

  26. Wang H, Huang M, Zhu X (2008) A generative probabilistic model for multi-label classification. Proceedings of the 8th IEEE international conference on data mining. IEEE, Pisa, Italy, pp 628–637

    Google Scholar 

  27. Boutell M, Luo J, Shen X, Brown C (2004) Learning multi-labels scene classification. Pattern Recognit 37(9):1757–1771

    Article  Google Scholar 

  28. Tanaka EA, Baranauskas JA (2012) An adaptation of binary relevance for multi-label classification applied to functional genomics. In: Proceedings of the workshop on medical informatics. Curitiba, Brazil, pp 1–10

  29. Cesa-Bianchi N, Gentile C, Zaniboni L et al (2006) Incremental algorithms for hierarchical classification. J Mach Learn Res 7(1):31–54

    MathSciNet  MATH  Google Scholar 

  30. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359

    Article  MathSciNet  Google Scholar 

  31. Read J (2008) A pruned problem transformation method for multi-label classification. In: Proceedings of New Zealand computer science research student conference. Christchurch, New Zealand, pp 143–150

  32. Read J, Pfahringer B, Holmes G (2008) Multi-label classification using ensembles of pruned sets. In: Proceedings of the 8th IEEE international conference on data mining. IEEE, Pisa, Italy, pp 995–1000

  33. Tsoumakas G, Vlahavas I (2007) Random k-label sets: an ensemble method for multi-label classification. In: Proceedings of the 18th European conference on machine learning. Springer, Warsaw, Poland, pp 406–417

  34. Hüllermeier E, Fürnkranz J, Cheng W, Brinker K (2008) Label ranking by learning pairwise preferences. Artif Intell 172(16):1897–1916

    Article  MathSciNet  MATH  Google Scholar 

  35. Fürnkranz J, Hüllermeier E, Mencía EL, Brinker K (2008) Multi-label classification via calibrated label ranking. Mach Learn 73(2):133–153

    Article  Google Scholar 

  36. Brinker K, Fürnkranz J, Hüllermeier E (2006) A unified model for multi-label classification and ranking. In: Proceedings of the 17th European conference on artificial intelligence. ACM, Italy, pp 489–493

  37. Elisseeff A, Weston J (2002) A kernel method for multi-labeled classification. In: Dietterich TG, Becker S, Ghahramani Z (eds) Advances in neural information processing systems, vol 14. MIT Press, Cambridge, pp 681–687

  38. Huang S-J, Zhou Z-H (2012) Multi-label learning by exploiting label correlations locally. In: Proceedings of the 26th AAAI conference on artificial intelligence (AAAI’12). ACM, Toronto, Canada, pp 949–955

  39. Trohidis K, Tsoumakas G, Kalliris G, Vlahavas I (2008) Multilabel classification of music into emotions. In: Proceedings of the 9th international Society for Music Information Retrieval. Philadelphia: ISMIR, pp 325–330

  40. Elisseeff A, Weston J (2002) A kernel method for multi-labeled classification. Advances in neural information processing systems, vol 14. MIT Press, Cambridge, pp 681–687

    Google Scholar 

  41. Sebastiani F (2002) Machine learning in automated text categorization. ACM Comput Surv 34(1):1–47

    Article  Google Scholar 

  42. Briggs F, Huang Y, Raich R, Eftaxias K (2013) The 9th annual MLSP competition: new methods for acoustic classification of multiple simultaneous bird species in a noisy environment. In: IEEE international workshop on machine learning for signal processing, pp 1–8

  43. Ueda N, Saito K (2003) Parametric mixture models for multi-labeled text. Advances in Neural information processing systems, vol 15. MIT Press, Cambridge, pp 721–728

    Google Scholar 

  44. Huang S (2014) Research on label relationship exploitation in multi-label learning. Ph.D. Thesis, Nanjing University

  45. Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification. In: Dietterich TG, Becker S, Ghahramani Z (eds) Advances in neural information processing systems, vol 14. MIT Press, Cambridge, pp 681–687

    Google Scholar 

  46. Qiu S, Lane T (2009) A framework for multiple kernel support vector regression and its applications to siRNA efficacy prediction. IEEE/ACM T Comput Biol Biol 6(2):190–199

    Google Scholar 

Download references

Acknowledgements

This work is supported by grants of NSFC (71471060) and Fundamental Research Funds for the Central Universities (2018ZD06).

Author information

Authors and Affiliations

  1. School of Control and Computer Engineering, North China Electric Power University, Beijing, 102206, China

    Linlin Chen

  2. School of Mathematics and Physics, North China Electric Power University, Beijing, 102206, China

    Degang Chen

  3. School of Computing and Mathematics, University of Ulster, Northern Ireland, BT370QB, UK

    Hui Wang

Authors
  1. Linlin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Degang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Linlin Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, L., Chen, D. & Wang, H. Alignment Based Kernel Selection for Multi-Label Learning. Neural Process Lett 49, 1157–1177 (2019). https://doi.org/10.1007/s11063-018-9863-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-018-9863-z

Keywords

Search

Navigation