Skip to main content
SpringerLink
Log in

Collective Mining of Bayesian Networks from Distributed Heterogeneous Data

  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

We present a collective approach to learning a Bayesian network from distributed heterogeneous data. In this approach, we first learn a local Bayesian network at each site using the local data. Then each site identifies the observations that are most likely to be evidence of coupling between local and non-local variables and transmits a subset of these observations to a central site. Another Bayesian network is learnt at the central site using the data transmitted from the local site. The local and central Bayesian networks are combined to obtain a collective Bayesian network, which models the entire data. Experimental results and theoretical justification that demonstrate the feasibility of our approach are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. 1991Abe:1991 Abe N, Takeuchi J, Warmuth M (1991) Polynomial learnability of probabilistic concepts with respect to the Kullback-Leibler divergence. In Proceedings of the 1991 Workshop on Computational Learning Theory, Santa Cruz, CA, pp 277–289

  2. 1997Bauer:1997 Bauer E, Koller D, Singer Y (1997) Update rules for parameter estimation in Bayesian networks. In Geiger D, Shanoy P (eds). Proceedings of the 13th conference on uncertainty in artificial intelligence. Morgan Kaufmann, Santa Mateo, CA, pp 3–13

  3. 1989Beinlich:1989 Beinlich I, Suermondt H, Chavez R, Cooper G (1989) The alarm monitoring system: a case study with two probabilistic inference techniques for belief networks. In Proceedings of the 2nd European conference on artificial intelligence in medical care. Springer, Berlin, pp 247–256

    Google Scholar 

  4. 1997Binder:1997 Binder J, Koller D, Russel S, Kanazawa K (1997) Adaptive probabilistic networks with hidden variables. Machine Learning 29: 213–244

    Article  Google Scholar 

  5. 1994Bouckaert:1994 Bouckaert RR (1994) Properties of Bayesian network learning algorithms. In de~Mantaras RL, Poole D (eds). Proceedings of the 10th conference on uncertainty in artificial intelligence. Morgan Kaufmann, Ssan Mateo, CA, pp 102–109

  6. 1998Breese:1998 Breese JS, Heckerman D, Kadie C, (1998) Empirical analysis of predictive algorithms for collaborative filtering. In Cooper GF, Moral S (eds). Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann, San Mateo, CA pp. 43–52

  7. 1991Buntine:1991 Buntine W (1991) Theory refinement on Bayesian networks. In D’Ambrosio BD, Bonissone PP, (eds) Proceedings of the 7th annual conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CA, pp 52–60

  8. 1996Cheeseman:1995 Cheeseman P, Stutz J (1996) Bayesian classification (autoclass): theory and results. In Fayyad U, Shapiro GP, Smyth P, Uthurasamy RS (eds). Advances in knowledge discovery and data mining. AAAI Press, Menlo Park, CA. pp. 153–180

  9. 2002Chen:2002 Chen R, Sivakumar K (2002) A new algorithm for learning parameters of a Bayesian network from distributed data. In Proceedings of the 2002 IEEE international conference on data mining. Maebashi City, Japan. pp. 585–588

  10. 2001aChen:2001a Chen R, Sivakumar K, Kargupta H (2001a) An approach to online Bayesian learning from multiple data streams. In Kargupta H, Sivakumar K, Wirth R (eds). Proceedings of the workshop on ubiquitous data mining: technology for mobile and distributed KDD (5th European conference, PKDD 2001). Freiburg, Germany, pp 31–45

    Google Scholar 

  11. 2001bChen:2001b Chen R, Sivakumar K, Kargupta H (2001b) Distributed web mining using Bayesian networks from multiple data streams. In Proceedings of the 2001 IEEE international conference on data mining. San Jose, CA. pp. 75–82

  12. 1997Cheng:1997 Cheng J, Bell DA, Liu W (1997) Learning belief networks from data: an information theory based approach. In proceedings of the 6th ACM international conference on information and knowledge management.

    Google Scholar 

  13. 1996Chickering:1996 Chickering DM (1996) Learning equivalence classes of Bayesian network structure. In Horvitz E, Jensen F (eds). Proceedings of the 12th conference on uncertainty in artificial intelligence, Morgan Kaufmann, San Mateo, CA

  14. 1997Chickering:1997 Chickering DM, Heckerman D (1997) Efficient approximation for the marginal likelihood of incomplete data given a Bayesian network. Machine Learning 29:181–212

    Article  Google Scholar 

  15. 1992Cooper:1992 Cooper GF, Herskovits E (1992) A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9: 309–347

    Article  Google Scholar 

  16. 1997Dasgupta:1997 Dasgupta S (1997) The sample complexity of learning fixed-structure Bayesian networks. Machine Learning 29: 165–180

    Article  Google Scholar 

  17. 1995Ezawa:1995 Ezawa KJ, Schuermann T (1995) Fraud/uncollectable debt detection using Bayesian network based learning system: a rare binary outcome with mixed data structures. In Besnard P, Hanks S (eds). Proceedings of the 11th conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CA, pp 157–166

  18. 1998Friedman:1998 Friedman N (1998) The Bayesian structural EM algorithm. In Cooper GF, Moral S, (eds). Proceedings of the 14th conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CA

  19. 1997Friedman:1997a Friedman N, Geiger D, Goldszmidt M (1997), Bayesian network classifiers. Machine Learning 29:131–163

    Article  Google Scholar 

  20. 1997Friedman:1997b Friedman N, Goldszmidt M (1997) Sequential update of Bayesian network structure. In Geiger D, Shanoy P (eds). Proceedings of the 13th conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CA

  21. 1996Friedman:96 Friedman N, Yakhini Z (1996) On the sample complexity of learning Bayesian networks. In Proceedings of the twelfth conference on uncertainty in artificial intelligence.

  22. 1996Gilks:1996 Gilks W, Richardson S, Spiegelhalter D (1996), Markov chain Monte Carlo in practice. Chapman Hall London.

  23. 1998Heckerman:1998 Heckerman D (1998) A tutorial on learning with Bayesian networks. In Jordan MI (ed). Proceedings of the NATO advanced study institute on learning in graphical models. Kluwer, Dordrecht

  24. 1995Heckerman:1995a Heckerman D, Geiger D, Chickering DM (1995) Learning Bayesian networks: the combination of knowledge and statistical data. Machine Learning 20:197–243

    Article  Google Scholar 

  25. 1995Heckerman:1995 Heckerman D, Gieger D (1995) Learning Bayesian networks: a unification for discrete and Gaussian domains. In Besnard P, Hanks S (eds). Proceedings of the 11th conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CA, pp 274–284

    Google Scholar 

  26. 2001Hershberger:01 Hershberger D, Kargupta H (2001), Distributed multivariate regression using wavelet-based collective data mining. Journal of Parallel and Distributed Computing 61:372–400

    Article  Google Scholar 

  27. 1996Jensen:1996 Jensen F (1996) An introduction to Bayesian networks, Springer, Berlin.

  28. 2001Kargupta:01 Kargupta H, Huang W, Sivakumar K, Johnson E (2001), Distributed clustering using collective principal component analysis. Knowledge and Information Systems Journal 3:422–448

    Google Scholar 

  29. 2000Kargupta:00c Kargupta H, Park B, Hershberger D, Johnson E (2000) Collective data mining: a new perspective toward distributed data mining. In Kargupta H, Chan P (eds). Advances in distributed and parallel knowledge discovery. AAAI/ MIT Press, Menlo Park, CA, pp 133–184

    Google Scholar 

  30. 1997Kenji:1997 Kenji Y (1997) Distributed cooperative Bayesian learning strategies. In Proceedings of the 10th annual conference on computational learning theory. ACM Press, Nashville, TN, pp 250–262

  31. 1994Lam:1994 Lam W, and Bacchus F, (1994) Learning Bayesian belief networks: an approach based on the MDL principle. Computational intelligence 10:262–293

    Google Scholar 

  32. 1995Lauritzen:1995 Lauritzen SL (1995) The EM algorithm for graphical association models with missing data. Computational Statistics and Data Analysis 19:191–201

    Article  Google Scholar 

  33. 1988Lauritzen:1988 Lauritzen SL, Spiegelhalter DJ (1988) Local computations with probabilities on graphical structures and their application to expert systems (with discussion). Journal of the Royal Statistical Society, series B 50:157–224

    Google Scholar 

  34. 1994Madigan:1994 Madigan D, Raftery A (1994) Model selection and accounting for model uncertainty in graphical models using Occam’s window. Journal of the American Statistical Association 89:1535–1546

    Google Scholar 

  35. 1998McLean:98 McLean B, Hawkins C, Spagna A, Lattanzi M, Lasker B, Jenkner H, White R (1998) New horizons from multi-wavelength sky surveys. IAU Symposium 179, Baltimore, MD

  36. 1998Meila:1998 Meila M, Jordan MI (1998) Estimating dependency structure as a hidden variable. In ‘NIPS’

  37. 2002Park:02 Park B, Kargupta H (2002) Distributed data mining: algorithms, systems, and applications. In Ye N (ed). Data mining handbook (To appear)

    Google Scholar 

  38. 1993Pearl:1993 Pearl J (1993) Graphical models. causality and intervention. Statistical Science 8:266–273

    Google Scholar 

  39. 1997Singh:1997 Singh M (1997) Learning Bayesian networks from incomplete data. In Proceedings of the national conference on artificial intelligence. AAAI Press Menlo Park, CA, pp 27–31

  40. 1995Singh:1995 Singh M, Provan GM (1995) A comparison of induction algorithms for selective and non-selective Bayesian classifiers. In Prieditis A, Russel S (eds). Proceedings of the 12th international conference on machine learning. Morgan Kaufmann, San Mateo, CA, pp 497–505

  41. 1990Spiegelhalter:1990 Spiegelhalter DJ, Lauritzen SL (1990) Sequential updating of conditional probabilities on directed graphical structures. Networks 20:570–605

    Google Scholar 

  42. 1993Spirtes:1993 Spirtes P, Glymour C, Scheines R, (1993) Causation, prediction and search. In Lecture notes in statistics. no 81. Springer, Berlin

  43. 1993Suzuki:1993 Suzuki J (1993) A construction of Bayesian networks from databases based on an MDL scheme. In Heckerman D, Mamdani A (eds). Proceedings of the 9th conference on uncertainty in artificial intelligence. Morgam Kaufmann, San Mateo, CA, pp 266–273

  44. 1998Szalay:98 Szalay A (1998) The evolving universe. ASSL (231)

  45. 1995Thiesson:1995 Thiesson B (1995) Accelerated quantification of Bayesian networks with incomplete data. In Proceedings of the first international conference on knowledge discovery and data mining. AAAI Press, Menlo Park, CA, pp 306–311

  46. 1992Thomas:1992 Thomas A, Spiegelhalter D, Gilks W (1992) Bugs: a program to perform Bayesian inference using Gibbs sampling. In Bernardo J, Berger J, Dawid A, Smith A (eds). Bayesian statistics. Oxford University Press, Oxford, pp 837–842.

  47. 1998Zweig:1998 Zweig G, Russel SJ (1998) Speech recognition with dynamic Bayesian networks. In Proceedings of the 15th national conference on artificial intelligence, Madison, WI

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computer Science, Washington State University, Pullman, WA, 99164-2752, USA

    R. Chen & K. Sivakumar

  2. Department of Computer Science and Electrical Engineering, University of Maryland Baltimore County, Baltimore, MD, USA

    H. Kargupta

Authors
  1. R. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. Sivakumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. Kargupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Sivakumar.

Additional information

16 November 2001

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, R., Sivakumar, K. & Kargupta, H. Collective Mining of Bayesian Networks from Distributed Heterogeneous Data. Knowledge and Information Systems 6, 164–187 (2004). https://doi.org/10.1007/s10115-003-0107-8

Download citation

  • Revised:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-003-0107-8

Keywords

Search

Navigation