Skip to main content
SpringerLink
Log in

Kernel extreme learning machine based on fuzzy set theory for multi-label classification

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Multi-label classification is a special kind of classification problem, where a single instance can be labeled to more than one class. Extreme learning machine (ELM) with kernel is an efficient method for solving both regression and multi-class classification problems. However, ELM with kernel has a limitation when it comes to multi-label classification tasks. To solve this problem, this paper proposes an enhanced ELM with kernel based on a fuzzy set theory for multi-label classification problems. The relationship between an instance and its corresponding class can be defined as the fuzzy membership. This fuzzy membership is used in output weights computation to weigh the training sample towards the corresponding classes. The experimental results demonstrate that the proposed method outperforms the ELM family of algorithms for multi-label problems, as well as the state-of-the-art multi-label classification algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. http://mulan.sourceforge.net/ datasets.html

  2. https://archive.ics.uci.edu/ml/index.php

References

  1. Hornik K (1991) Approximation capabilities of multilayer feedforward networks. Neural Netw 4(2):251–257

    Article  MathSciNet  Google Scholar 

  2. Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 2(4):303–314

    Article  MathSciNet  MATH  Google Scholar 

  3. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366

    Article  MATH  Google Scholar 

  4. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489–501

    Article  Google Scholar 

  5. Huang GB, Chen L, Siew CK et al (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  6. Rong HJ, Huang GB, Ong YS (2008) Extreme learning machine for multi-categories classification applications. In: Proceedings of the international joint conference on neural networks, pp 1709–1713

  7. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B (Cybernetics) 42(2):513–529

    Article  Google Scholar 

  8. Tsoumakas G, Katakis I (2007) Multi-label classification: an overview. Int J Data Warehous Min 3(3):1–13

    Article  Google Scholar 

  9. Madjarov G, Kocev D, Gjorgjevikj D, Džeroski S (2012) An extensive experimental comparison of methods for multi-label learning. Pattern Recognit 45(9):3084–3104

    Article  Google Scholar 

  10. Gibaja E, Ventura S (2015) A tutorial on multilabel learning. ACM Comput Surv 47(3):52

    Article  Google Scholar 

  11. Zhang Y, Schneider J (2011) Multi-label output codes using canonical correlation analysis. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 873–882

  12. Hotelling H (1936) Relations between two sets of variates. Biometrika 28(3/4):321–377

    Article  MATH  Google Scholar 

  13. Zhang ML, Zhou ZH (2007) Ml-knn: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048

    Article  MATH  Google Scholar 

  14. Zadeh LA (1965) Fuzzy sets. Inf Control 8(3):338–353

    Article  MATH  Google Scholar 

  15. Clare A, King R (2001) Knowledge discovery in multi-label phenotype data. Principles of data mining and knowledge discovery pp 42–53

  16. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359

    Article  MathSciNet  Google Scholar 

  17. Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. Mach Learn ECML 2007:406–417

    Google Scholar 

  18. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771

    Article  Google Scholar 

  19. Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification. In: Advances in neural information processing systems, pp 681–687

  20. Zhang ML, Zhou ZH (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351

    Article  Google Scholar 

  21. Zhang ML (2009) Ml-rbf: Rbf neural networks for multi-label learning. Neural Process Lett 29(2):61–74

    Article  Google Scholar 

  22. Zimmermann HJ (2010) Fuzzy set theory. Wiley Interdiscip Rev Comput Stat 2(3):317–332

    Article  MathSciNet  Google Scholar 

  23. Zimmermann HJ (2011) Fuzzy set theory and its applications. Springer, New York

    Google Scholar 

  24. Werro N (2015) Fuzzy classification of online customers. Springer, New York

    Book  Google Scholar 

  25. Zhai J, Zhang Y, Zhu H (2017) Three-way decisions model based on tolerance rough fuzzy set. Int J Mach Learn Cybern 8(1):35–43

    Article  Google Scholar 

  26. Scherer R (2012) Multiple fuzzy classification systems, vol 288. Springer, New York

    MATH  Google Scholar 

  27. Shin YC, Xu C (2008) Intelligent systems: modeling, optimization, and control, vol 30. CRC Press, Boca Raton

    Google Scholar 

  28. Sumathi S, Paneerselvam S (2010) Computational intelligence paradigms: theory and applications using MATLAB. CRC Press, Boca Raton

    Google Scholar 

  29. Dubois D, Prade H (2012) Fundamentals of fuzzy sets, vol 7. Springer, New York

    MATH  Google Scholar 

  30. Belohlavek R, Klir GJ (2011) Concepts and fuzzy logic. MIT Press, Cambridge

    Book  MATH  Google Scholar 

  31. Türksen IB (2005) An ontological and epistemological perspective of fuzzy set theory. Elsevier, Amsterdam

    MATH  Google Scholar 

  32. Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16):3056–3062

    Article  Google Scholar 

  33. Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16):3460–3468

    Article  Google Scholar 

  34. Zhai J, Li T, Wang X (2016) A cross-selection instance algorithm. J Intell Fuzzy Syst 30(2):717–728

    Article  Google Scholar 

  35. Zhai Jh, Hy Xu, Xz Wang (2012) Dynamic ensemble extreme learning machine based on sample entropy. Soft Comput A Fusion Found Methodol Appl 16(9):1493–1502

    Google Scholar 

  36. Bi X, Zhao X, Wang G, Zhang P, Wang C (2015) Distributed extreme learning machine with kernels based on mapreduce. Neurocomputing 149:456–463

    Article  Google Scholar 

  37. Zhai J, Wang X, Pang X (2016) Voting-based instance selection from large data sets with mapreduce and random weight networks. Inf Sci 367:1066–1077

    Article  Google Scholar 

  38. Feng G, Lan Y, Zhang X, Qian Z (2015) Dynamic adjustment of hidden node parameters for extreme learning machine. IEEE Trans Cybern 45(2):279–288

    Article  Google Scholar 

  39. Zhai J, Shao Q, Wang X (2016) Architecture selection of elm networks based on sensitivity of hidden nodes. Neural Process Lett 44(2):471–489

    Article  Google Scholar 

  40. Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122

    Article  Google Scholar 

  41. Huang G, Huang GB, Song S, You K (2015) Trends in extreme learning machines: a review. Neural Netw 61:32–48

    Article  MATH  Google Scholar 

  42. Zhu H, Tsang EC, Wang XZ, Aamir Raza Ashfaq R (2017) Monotonic classification extreme learning machine. Neurocomputing 225:205–213

    Article  Google Scholar 

  43. Zhang W, Ji H (2013) Fuzzy extreme learning machine for classification. Electr Lett 49(7):448–450

    Article  Google Scholar 

  44. Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(2):464–471

    Article  Google Scholar 

  45. Coufal D (2017) Radial fuzzy systems. Fuzzy Sets Syst 319:1–27

    Article  MathSciNet  MATH  Google Scholar 

  46. de Barros JC, Dexter AL (2007) On-line identification of computationally undemanding evolving fuzzy models. Fuzzy Sets Syst 158(18):1997–2012

    Article  MathSciNet  MATH  Google Scholar 

  47. Zong W, Huang GB, Chen Y (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Graduate Education of Computer and Information Science Interdisciplinary Research Grant from Department of Computer Science, Khon Kaen University (Grant no. 001/2558).

Author information

Authors and Affiliations

  1. Department of Computer Science, Khon Kaen University, Mittraphap Road, Khon Kaen, 40002, Thailand

    Yanika Kongsorot, Punyaphol Horata, Pakarat Musikawan & Khamron Sunat

Authors
  1. Yanika Kongsorot
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Punyaphol Horata
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pakarat Musikawan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Khamron Sunat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Punyaphol Horata.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kongsorot, Y., Horata, P., Musikawan, P. et al. Kernel extreme learning machine based on fuzzy set theory for multi-label classification. Int. J. Mach. Learn. & Cyber. 10, 979–989 (2019). https://doi.org/10.1007/s13042-017-0776-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-017-0776-3

Keywords

Search

Navigation