Skip to main content
Springer Nature Link
Log in

Learning Bayesian network parameters with soft-hard constraints

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In Bayesian network parameter learning, it is difficult to obtain accurate parameters when the data are insufficient, and overfitting easily occurs. However, underfitting is prone to happen when the learning results are blindly close to the constraints generated by expert knowledge. A soft-hard constraint parameter learning method is proposed to balance the overfitting and underfitting problems in parameter learning. In this paper, the constraints applied to the parameters are called hard constraints, while those used to the prior are called soft constraints. A partial maximum entropy prior quantization method is proposed for the soft constraint to obtain proper parameters. The equivalent sample threshold is proposed to limit the hyperparameters based on the actual data size for the hard constraint. To combine soft and hard constraints effectively, the soft constraint maximization and hard constraint minimization model is proposed. Experimental results show that this method can significantly improve the accuracy of parameter learning and actually balance overfitting and underfitting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://github.com/bayesnet/bnt/tree/master/BNT.

  2. http://www.bnlearn.com/bnrepository.

  3. https://github.com/RURURU24/BN.git.

References

  1. Pearl J (1988) probabilistic reasoning in intelligent systems: networks of plausible inference San Mateo. Comput Sci Artif Intell 58(2):721

    Google Scholar 

  2. Chen G, Ge Z (2020) Robust bayesian networks for low-quality data modeling and process monitoring applications. Control Eng Pract 97(104344):1–14

    Google Scholar 

  3. Zhang Y, Weng WG (2020) Bayesian network model for buried gas pipeline failure analysis caused by corrosion and external interference. Reliabil Eng Syst Safety 203:107089

    Article  Google Scholar 

  4. Fg A, Nm B (2020) A new scoring system for the rapid entire body assessment (reba) based on Fuzzy sets and Bayesian networks. Int J Ind Ergon 80:103058

    Article  Google Scholar 

  5. Zhang GH, Chen W, Jiao YY, Wang H, Wang CT (2020) A failure probability evaluation method for collapse of drill-and-blast tunnels based on multistate Fuzzy Bayesian network. Eng Geol 276(9):105752

    Article  Google Scholar 

  6. Zhang T, Zhang T, Li C, Zhai X, Huo Q (2021) Complementary and alternative therapies for precancerous lesions of gastric cancer: a protocol for a Bayesian network meta analysis. Medicine 100(2):24249

    Article  Google Scholar 

  7. Shree SRB, Sheshadri HS (2018) Diagnosis of Alzheimer’s disease using naive Bayesian classifier. Neural Comput Appl 29(3):123–132

    Article  Google Scholar 

  8. Kaghazchi A, Shahdany SH, Roozbahani A (2021) Simulation and evaluation of agricultural water distribution and delivery systems with a hybrid bayesian network model. Agric Water Manage 245(c):106578

  9. Alkheder S, Alkhamees W, Almutairi R, Alkhedher M (2021) Bayesian combined neural network for traffic volume short-term forecasting at adjacent intersections. Neural Comput Appl 33:1785–1836

    Article  Google Scholar 

  10. Redner RA (1984) Mixture densities, maximum likelihood and the em algorithm. Siam Rev 26(2):195–239

    Article  MathSciNet  Google Scholar 

  11. Yang Y, Gao XG, Guo ZG (2015) Learning bn parameters with small data sets based by data reutilization. Acta Auto Sin 41(12):14

    MATH  Google Scholar 

  12. Xgg A, Zgg A, Hao RA, Yu YA, Dqc B, Cch A (2019) Learning bayesian network parameters via minimax algorithm. Int J Approx Reason 108:62–75

    Article  MathSciNet  Google Scholar 

  13. Xt A, Xg A, Zw A, Cha B (2021) Bidirectional heuristic search to find the optimal Bayesian network structure. Neurocomputing 41(426):35–46

    Google Scholar 

  14. Dai J, Ren J, Du W, Vladimir S, Ma J (2020) An improved evolutionary approach-based hybrid algorithm for Bayesian network structure learning in dynamic constrained search space. Neural Comput Appl 32:1413–1434

    Article  Google Scholar 

  15. Zw A, Xg A, Xt A, Yu YB, Dc C (2021) Learning Bayesian networks based on order graph with ancestral constraints. Knowl Based Syst 211:106515

    Article  Google Scholar 

  16. Dempster AP (1977) Maximum likelihood from incomplete data via the em algorithm. J R Stat Soc 39(1):1–38

    MathSciNet  MATH  Google Scholar 

  17. Liao W, Qiang J (2009) Learning Bayesian network parameters under incomplete data with domain knowledge. Pattern Recogn 42(11):3046–3056

    Article  Google Scholar 

  18. (2020) Learning parameters of bayesian networks from datasets with systematically missing data: a meta-analytic approach. Exp Syst Appl 141:112956

  19. Guo ZG, Gao XG, Di RH (2014) Learning Bayesian network parameters under dual constraints from small data set. Acta Auto Sin 40(7):1509–1516

    Google Scholar 

  20. Yun Z, Norman F, Cheng Z (2016) An empirical study of bayesian network parameter learning with monotonic influence constraints. Decision Support Syst 87(C):69–79

  21. Campos C, Yan T, Qiang J (2008) Constrained maximum likelihood learning of bayesian networks for facial action recognition. In: Computer Vision-ECCV 2008, pp 168–181

  22. Feelders A, Gaag L (2006) Learning Bayesian network parameters under order constraints. Int J Approx Reason 42:37–53

    Article  MathSciNet  Google Scholar 

  23. Feelders A, Gaag L (2012) Learning bayesian network parameters with prior knowledge about context-specific qualitative influences. comput Sci, pp 193–200

  24. Yang Y, Gao X, Guo Z, Chen D (2019) Learning Bayesian networks using the constrained maximum a posteriori probability method. Pattern Recogn 91:123–134

    Article  Google Scholar 

  25. Chang R, Wang W (2010) Novel algorithm for bayesian network parameter learning with informative prior constraints. In: The 2010 international joint conference on neural networks (IJCNN), pp 1–8

  26. Guo ZG, Gao XG, Ren H, Yang Y, Di RH, Chen DQ (2017) Learning Bayesian network parameters from small data sets: a further constrained qualitatively maximum a posteriori method. Int J Approx Reason 91(09):22–35

    Article  MathSciNet  Google Scholar 

  27. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79–86

    Article  MathSciNet  Google Scholar 

  28. Dasgupta S (1997) The sample complexity of learning fixed-structure Bayesian networks. Mach Learn 29(2):165–180

    Article  Google Scholar 

  29. Clarke BS, Barron AR (1994) Jeffreys’ prior is asymptotically least favorable under entropy risk. J Stat Plan Inference 41(1):37–60

    Article  MathSciNet  Google Scholar 

  30. Suzuki J (1996) Learning bayesian belief networks based on the minimum description length principle: basic properties. Ieice Trans Fund 82(10):2237–2245

  31. Lja B, Lz A, Ly C, Dw C (2019) Class-specific attribute weighted naive bayes. Pattern Recogn 88:321–330

    Article  Google Scholar 

  32. Jiang L, Zhang L, Li C, Wu J (2019) A correlation-based feature weighting filter for naive bayes. IEEE Trans Knowl Data Eng 21(2):201–213

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China (61573285).

Author information

Authors and Affiliations

  1. School of Electronics and Information, Northwestern Polytechnical University, Dongxiang Road, Xi’an, 710072, Shaanxi, China

    Xinxin Ru, Xiaoguang Gao, Yangyang Wang & Xiaohan Liu

Authors
  1. Xinxin Ru
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiaoguang Gao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yangyang Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xiaohan Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xiaoguang Gao.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ru, X., Gao, X., Wang, Y. et al. Learning Bayesian network parameters with soft-hard constraints. Neural Comput & Applic 34, 18195–18209 (2022). https://doi.org/10.1007/s00521-022-07429-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07429-5

Keywords