Skip to main content
SpringerLink
Log in

Incremental deep forest for multi-label data streams learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-label learning has attracted many attentions. However, the continuous data generated in the fields of sensors, network access, etc., that is data streams, the scenario brings challenges such as real-time, limited memory, once pass. Several learning algorithms have been proposed for offline multi-label classification, but few researches develop it for dynamic multi-label incremental learning models based on cascading schemes. Deep forest can perform representation learning layer by layer, and does not rely on backpropagation, using this cascading scheme, this paper proposes a multi-label data stream deep forest (VDSDF) learning algorithm based on cascaded Very Fast Decision Tree (VFDT) forest, which can receive examples successively, perform incremental learning, and adapt to concept drift. Experimental results show that the proposed VDSDF algorithm, as an incremental classification algorithm, is more competitive than batch classification algorithms on multiple indicators. Moreover, in dynamic flow scenarios, the adaptability of VDSDF to concept drift is better than that of the contrast algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Liu W, Wang H, Shen X, Tsang I (2021) The emerging trends of multi-label learning. IEEE Transactions on Pattern Analysis and Machine Intelligence

  2. Balakrishna S, Thirumaran M, Solanki VK, Núñez Valdéz ER, et al. (2020) Incremental hierarchical clustering driven automatic annotations for unifying iot streaming data. International Journal Of Interactive Multimedia And Artificial Intelligence

  3. Moyano JM, Gibaja EL, Cios KJ, Ventura S (2018) Review of ensembles of multi-label classifiers: models, experimental study and prospects. Information Fusion 44:33–45

    Article  Google Scholar 

  4. Yan M (2020) Accurate detecting concept drift in evolving data streams. ICT Express

  5. Cerri R, Júnior JDC, Paiva ERF, da Gama JMP (2020) Multi-label stream classification with self-organizing maps. arXiv preprint arXiv:2004.09397

  6. Wang L, Lin X, Lin N (2021) Research on pseudo-label technology for multi-label news classification. In: International conference on document analysis and recognition, Springer, pp 683–698

  7. Cerri R, Basgalupp MP, Barros RC, de Carvalho ACPLF (2019) Inducing hierarchical multi-label classification rules with genetic algorithms. Appl Soft Comput

  8. Pliakos K, Geurts P, Vens C (2018) Global multi-output decision trees for interaction prediction. Mach Learn 107(8):1257–1281

    Article  MathSciNet  Google Scholar 

  9. Wang L, Liu Y, Qin C, Sun G, Fu Y (2020) Dual relation semi-supervised multi-label learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 6227–6234

  10. Gong X, Yuan D, Bao W (2020) Online metric learning for multi-label classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 4012–4019

  11. Mehta S et al (2017) Concept drift in streaming data classification: algorithms, platforms and issues. Procedia computer science 122:804–811

    Article  Google Scholar 

  12. Yang L, Wu X-Z, Jiang Y, Zhou Z-H (2019) Multi-label learning with deep forest. arXiv:1911.06557

  13. Yapp EKY, Li X, Lu WF, Tan PS (2020) Comparison of base classifiers for multi-label learning. Neurocomputing 394:51–60

    Article  Google Scholar 

  14. Zhang C, Li Z (2021) Multi-label learning with label-specific features via weighting and label entropy guided clustering ensemble. Neurocomputing 419:59–69

    Article  Google Scholar 

  15. Rodrigues J, Cristo M, Colonna JG (2020) Deep hashing for multi-label image retrieval: a survey. Artif Intell Rev 53(7):5261–5307

    Article  Google Scholar 

  16. Gomes HM, Bifet A, Read J, Barddal JP, Enembreck F, Pfharinger B, Holmes G, Abdessalem T (2017) Adaptive random forests for evolving data stream classification. Mach Learn 106(9):1469–1495

    Article  MathSciNet  Google Scholar 

  17. Wang H, Abraham Z (2015) Concept drift detection for streaming data. In: 2015 international joint conference on neural networks (IJCNN), IEEE, pp 1–9

  18. Luong AV, Nguyen TT, Liew A W-C (2020) Streaming active deep forest for evolving data stream classification. arXiv:2002.11816

  19. Wang W, Dai Q, Li F, Xiong Y, Wei D-Q (2021) Mlcdforest: multi-label classification with deep forest in disease prediction for long non-coding rnas. Brief Bioinform 22(3):bbaa104

    Article  Google Scholar 

  20. Zhou Z-H, Feng J (2017) Deep forest: Towards an alternative to deep neural networks. arXiv:1702.08835

  21. Gibaja E, Ventura S (2014) Multi-label learning: a review of the state of the art and ongoing research. Wiley Interdisciplinary Reviews: Data Knowl Eng 4(6):411–444

    Google Scholar 

  22. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333

    Article  MathSciNet  Google Scholar 

  23. Zhang M-L, Zhou Z-H (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  24. Zhang M-L, Zhou Z-H (2007) Ml-knn: A lazy learning approach to multi-label learning. Pattern Recog 40(7):2038–2048

    Article  Google Scholar 

  25. Yixiao H, Ming P, Yuan J (2020) Mondrian deep forest. J Comput Res Dev 57(8):1594

    Google Scholar 

  26. Sheng-Jun H, Wei G, Zhi-Hua Z (2020) Fast multi-instance multi-label learning. IEEE Transactions on Pattern Analysis and Machine Intelligence

  27. Read J, Bifet A, Holmes G, Pfahringer B (2012) Scalable and efficient multi-label classification for evolving data streams. Mach Learn 88(1-2):243–272

    Article  MathSciNet  Google Scholar 

  28. Kocev D, Vens C, Struyf J, Džeroski S (2007) Ensembles of multi-objective decision trees. In: European conference on machine learning, Springer, pp 624–631

  29. Garcıa-Mart?n E, Lavesson N, Grahn H, Casalicchio E, Boeva V (2018) Hoeffding trees with nmin adaptation. In: IEEE

  30. Venkatesan R, Er MJ, Wu S, Pratama M (2016) A novel online real-time classifier for multi-label data streams. In: 2016 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1833–1840

  31. Zhu W, Miao J, Qing L (2015) Constrained extreme learning machines: A study on classification cases. Computer Science

  32. Hong S, Xiao C, Ma T, Li H, Sun J (2019) Mina: Multilevel knowledge-guided attention for modeling electrocardiography signals. In: Twenty-eighth international joint conference on artificial intelligence IJCAI-19

  33. Verma KK, Singh BM (2021) Deep multi-model fusion for human activity recognition using evolutionary algorithms. International Journal of Interactive Multimedia & Artificial Intelligence, 7(2)

  34. Kocev D, Vens C, Struyf J, Džeroski S (2013) Tree ensembles for predicting structured outputs. Pattern Recogn 46(3):817–833

    Article  Google Scholar 

  35. Read J, Reutemann P, Pfahringer B, Holmes G (2016) Meka: a multi-label/multi-target extension to weka

  36. Tsoumakas G, Vlahavas I (2007) Random k-labelsets: An ensemble method for multilabel classification. In: European conference on machine learning, Springer, pp 406–417

  37. Zhang Q-W, Zhong Y, Zhang M-L (2018) Feature-induced labeling information enrichment for multi-label learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32

  38. Wu X-Z, Zhou Z-H (2017) A unified view of multi-label performance measures. In: International Conference on Machine Learning, PMLR, pp 3780–3788

  39. Wang S, Minku LL, Ghezzi D, Caltabiano D, Tino P, Yao X (2013) Concept drift detection for online class imbalance learning. In: The 2013 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1–10

  40. Katakis I, Tsoumakas G, Vlahavas IP (2008) An ensemble of classifiers for coping with recurring contexts in data streams.. In: ECAI, pp 763–764

  41. Xia Y, Chen K, Yang Y (2021) Multi-label classification with weighted classifier selection and stacked ensemble. Inf Sci 557:421–442

    Article  MathSciNet  Google Scholar 

  42. Li P, Wang H, Böhm C, Shao J (2020) Online semi-supervised multi-label classification with label compression and local smooth regression.. In: IJCAI, pp 1359–1365

Download references

Acknowledgements

This work was supported in part by the Hebei Natural Science Foundation No.G2021203010 and No.F2021203038 in China, and a project supported by Key Laboratory of Robotics and Intelligent Equipment of Guangdong Regular Institutions of Higher EducationGrant No.2017KSYS009.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yanshan University, Qinhuangdao, Hebei Province, 066004, China

    Shunpan Liang, Weiwei Pan, Dianlong You & Ze Liu

  2. Dongguan University of Technology, Dongguan, Guangdong Province, 523808, China

    Ling Yin

Authors
  1. Shunpan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiwei Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dianlong You
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ze Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ling Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shunpan Liang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Big Data-Driven Large-Scale Group Decision Making Under Uncertainty

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, S., Pan, W., You, D. et al. Incremental deep forest for multi-label data streams learning. Appl Intell 52, 13398–13414 (2022). https://doi.org/10.1007/s10489-022-03414-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03414-6

Keywords

Search

Navigation