Skip to main content

Advertisement

Springer Nature Link
Log in

Fast Shrinking parents-children learning for Markov blanket-based feature selection

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

High-dimensional data leads to degraded performance of machine learning algorithms and weak generalization of models, so feature selection is of great importance. In a Bayesian network (BN), the Markov blanket (MB) of a target node (T) is the best feature subset of that node. Therefore, this paper proposes Fast Shrinking parents-children learning for Markov blanket-based feature selection (FSMB), which first determines the parents-children of the target node, and then discovers spouses while checking candidate parents-children set. In spouse determination process, a secondary screening strategy is proposed to remove false-positive spouses effectively. In this process, once the spouses of T with respect to T’s child are determined, the parents-children set is immediately tested and the false-positive parents-children are removed in time, which not only can avoid the influence of false-positive parents-children on the subsequent spouse discovery, but also do not need to determine the spouses for false-positive parents-children. To verify the effectiveness of FSMB, experiments were performed on eight state-of-the-art MB algorithms on six standard networks and eight real datasets, the results show that FSMB outperforms other algorithms in terms of accuracy and efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data availability

The data generated during the current study are available from the corresponding author on reasonable request.

Notes

  1. https://github.com/z-dragonl/Causal-Learner.

  2. https://www.bnlearn.com/bnrepository.

  3. http://www.dsl-lab.org/supplements/mmhc_paper/mmhc_index.html.

  4. https://archive.ics.uci.edu/.

References

  1. Akkem Y, Biswas SK, Varanasi A (2023) Smart farming using artificial intelligence: A review. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2023.105899

    Article  Google Scholar 

  2. Zhang C, Xu J, Tang R, Yang J, Wang W, Yu X, Shi S (2023) Novel research and future prospects of artificial intelligence in cancer diagnosis and treatment. J Hematol Oncol. https://doi.org/10.1186/s13045-023-01514-5

    Article  Google Scholar 

  3. Akkem Y, Biswas SK, Varanasi A (2023) Smart farming monitoring using ml and mlops. In: Hassanien, A.E., Castillo, O., Anand, S., Jaiswal, A. (eds.) International Conference on Innovative Computing and Communications, pp. 665–675. Springer, Singapore. https://doi.org/10.1007/978-981-99-3315-0_51

  4. Li J, Cheng K, Wang S, Morstatter F, Trevino RP, Tang J, Liu H (2018) Feature selection: A data perspective. ACM Comput Surv. https://doi.org/10.1145/3136625

    Article  Google Scholar 

  5. Cai J, Luo J, Wang S, Yang S (2018) Feature selection in machine learning: A new perspective. Neurocomputing 300:70–79. https://doi.org/10.1016/j.neucom.2017.11.077

    Article  Google Scholar 

  6. Yu K, Guo X, Liu L, Li J, Wang H, Ling Z, Wu X (2020) Causality-based feature selection: Methods and evaluations. ACM Comput Surv. https://doi.org/10.1145/3409382

    Article  Google Scholar 

  7. Sun L, Yang Y, Liu Y, Ning T (2022) Feature selection based on a hybrid simplified particle swarm optimization algorithm with maximum separation and minimum redundancy. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-022-01663-y

    Article  Google Scholar 

  8. Neufeld E (1993) Review: Judea pearl, probabilistic reasoning in intelligent systems: Networks of plausible inference. Journal of Symbolic Logic

  9. Yu K, Liu L, Li J (2021) A unified view of causal and non-causal feature selection. ACM Trans Knowl Discov Data. https://doi.org/10.1145/3436891

    Article  Google Scholar 

  10. Khaire UM, Dhanalakshmi R (2022) Stability of feature selection algorithm: A review. JOURNAL OF KING SAUD UNIVERSITY-COMPUTER AND INFORMATION SCIENCES 34(4):1060–1073. https://doi.org/10.1016/j.jksuci.2019.06.012

    Article  Google Scholar 

  11. Aliferis CF, Statnikov A, Tsamardinos I, Mani S, Koutsoukos XD (2010) Local causal and markov blanket induction for causal discovery and feature selection for classification part ii: Analysis and extensions. J Mach Learn Res 11(10):235–284

    MathSciNet  Google Scholar 

  12. Guo R, Cheng L, Li J, Hahn PR, Liu H (2020) A survey of learning causality with data: Problems and methods. ACM Comput Surv. https://doi.org/10.1145/3397269

  13. Liu W, Jiang Y, Peng L, Sun X, Gan W, Zhao Q, Tang H (2022) Inferring gene regulatory networks using the improved markov blanket discovery algorithm. Interdiscip Sci Comput Life Sci 14(1):168–181. https://doi.org/10.1007/s12539-021-00478-9

    Article  Google Scholar 

  14. Han B, Chen X-w, Talebizadeh Z (2011) Fepi-mb: identifying snps-disease association using a markov blanket-based approach. BMC BIOINFORMATICS 12(12). https://doi.org/10.1186/1471-2105-12-S12-S3. 9th International Workshop on Data Mining in Bioinformatics (BIOKDD),Washington, DC, JUL 25, 2010

  15. Buschur KL, Chikina M, Benos PV (2020) Causal network perturbations for instance-specific analysis of single cell and disease samples. Bioinformatics 36(8):2515–2521. https://doi.org/10.1093/bioinformatics/btz949

    Article  Google Scholar 

  16. Javed K, Maruf S, Babri HA (2015) A two-stage markov blanket based feature selection algorithm for text classification. Neurocomputing 157:91–104. https://doi.org/10.1016/j.neucom.2015.01.031

    Article  Google Scholar 

  17. Cervantes Salgado M, Pinto Elias R, Magadan Salazar A (2021) Function to flatten gesture data for specific feature selection methods to improve classification. Traitement Du Signal 38(4), 929–935. https://doi.org/10.18280/ts.380402

  18. Niinimaki T, Parviainen P (2012) Local Structure Discovery in Bayesian Networks. https://doi.org/10.48550/arXiv.1210.4888

  19. Gao T, Ji Q (2017) Efficient score-based Markov blanket discovery. Int. J. Approx. Reason. 80:277–293. https://doi.org/10.1016/j.ijar.2016.09.009

    Article  MathSciNet  Google Scholar 

  20. Sun L, Yang Y, Ning T (2023) A novel feature selection using markov blanket representative set and particle swarm optimization algorithm. Comput Appl Math. https://doi.org/10.1007/s40314-023-02221-0

    Article  MathSciNet  Google Scholar 

  21. Yang S, Guo X, Yu K, Huang X, Jiang T, He J, Gu L (2023) Causal feature selection in the presence of sample selection bias. ACM Trans Intell Syst Technol. https://doi.org/10.1145/3604809

    Article  Google Scholar 

  22. Margaritis D, Thrun S (2000) Bayesian network induction via local neighborhoods. Adv Neural Inform Process Syst 12:505–511

    Google Scholar 

  23. Tsamardinos I, Aliferis CF (2002) Towards principled feature selection: Relevancy, filters and wrappers

  24. Guo X, Yu K, Cao F, Li P, Wang H (2022) Error-aware markov blanket learning for causal feature selection. Inform Sci. https://doi.org/10.1016/j.ins.2021.12.118

    Article  Google Scholar 

  25. Tsamardinos I, Aliferis CF, Statnikov A (2003) Time and sample efficient discovery of markov blankets and direct causal relations. Knowl Discov Data Mining. doi 10(1145/956750):956838

  26. Aliferis CF, Tsamardinos I, Statnikov A (2003) Hiton: A novel markov blanket algorithm for optimal variable selection. In: Amia .annual Symposium Proceedings, pp. 21–25

  27. Wang H, Ling Z, Yu K, Wu X (2020) Towards efficient and effective discovery of markov blankets for feature selection. Inform Sci 509:227–242. https://doi.org/10.1016/j.ins.2019.09.010

    Article  Google Scholar 

  28. Guo X, Yu K, Liu L, Cao F, Li J (2022) Causal feature selection with dual correction. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2022.3178075

  29. Ling Z, Li B, Zhang Y, Wang Q, Yu K, Wu X (2023) Causal feature selection with efficient spouses discovery. IEEE Trans Big Data 9(2):555–568. https://doi.org/10.1109/TBDATA.2022.3178472

    Article  Google Scholar 

  30. Tsamardinos I, Aliferis CF, Statnikov AR (2003) Algorithms for large scale markov blanket discovery. In: International Flairs Conference

  31. Yaramakala S, Margaritis D (2006) Speculative markov blanket discovery for optimal feature selection. In: Fifth IEEE International Conference on Data Mining (ICDM’05). https://doi.org/10.1109/ICDM.2005.134

  32. Pea JM, Nilsson R, Bjrkegren J, Tegnér J (2007) Towards scalable and data efficient learning of markov boundaries. Int J Approx Reason 45(2):211–232. https://doi.org/10.1016/j.ijar.2006.06.008

    Article  Google Scholar 

  33. R. De Morais S, Aussem A (2008) A novel scalable and data efficient feature subset selection algorithm. In: Proceedings of the European Conference Machine Learning and Knowledge Discovery in Databases, pp. 298–312. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87481-2_20

  34. Gao T, Ji Q (2017) Efficient markov blanket discovery and its application. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2016.2539338

    Article  Google Scholar 

  35. Ling Z, Yu K, Wang H, Liu L, Ding W, Wu X (2019) Bamb: A balanced markov blanket discovery approach to feature selection. ACM Trans Intell Syst 10(5):52–15225. https://doi.org/10.1145/3335676

    Article  Google Scholar 

  36. Koller D, Sahami M (1996) Toward optimal feature selection. In: Proceedings of the Thirteenth International Conference on International Conference on Machine Learning. ICML’96, pp. 284–292. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA

  37. Yang X, Wang Y, Ou Y, Tong Y (2019) Three-fast-inter incremental association markov blanket learning algorithm. Pattern Recogn Lett 122:73–78. https://doi.org/10.1016/j.patrec.2019.02.002

    Article  Google Scholar 

  38. Fu S, Desmarais MC (2008) Fast Markov blanket discovery algorithm via local learning within single pass. In: Advances in Artificial Intelligence 21st Conference of the Canadian Society for Computational Studies of Intelligence, pp. 96–107. https://doi.org/10.1007/978-3-540-68825-9_10

  39. Pearl J (1988) probabilistic reasoning in intelligent systems-networks of plausible inference. morgan kaufmann

  40. Kui Y, Lin L, Jiuyong L, Huanhuan C (2018) Mining markov blankets without causal sufficiency. IEEE Trans Neural Netw Learn Syst 29:6333–6347. https://doi.org/10.1109/TNNLS.2018.2828982

    Article  MathSciNet  Google Scholar 

Download references

Funding

This work was supported by the National Key R &D Program of China (No.2019YFB1707301).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yanshan University, Qinhuangdao, 066004, Hebei, China

    Haoran Liu, Qianrui Shi, Yanbin Cai, Niantai Wang & Liyue Zhang

  2. INSA Centre Val de Loire, Université d’Orléans, PRISME, EA4229, Bourges Cedex, 18020, France

    Dayan Liu

  3. The Key Laboratory for Special Fiber and Fiber Sensor of Hebei Province, Yanshan University, Qinhuangdao, 066004, Hebei, China

    Haoran Liu, Qianrui Shi, Yanbin Cai, Niantai Wang & Liyue Zhang

Authors
  1. Haoran Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Qianrui Shi
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yanbin Cai
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Niantai Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Liyue Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Dayan Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Qianrui Shi.

Ethics declarations

Conflict of interest

The authors declare that they have no competing financial interests exist.

Ethical approval

This declaration is not applicable for the current study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, H., Shi, Q., Cai, Y. et al. Fast Shrinking parents-children learning for Markov blanket-based feature selection. Int. J. Mach. Learn. & Cyber. 15, 3553–3566 (2024). https://doi.org/10.1007/s13042-024-02108-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-024-02108-4

Keywords