Divide-and-Conquer Strategy for Large-Scale Dynamic Bayesian Network Structure Learning

Abstract

Dynamic Bayesian Networks (DBNs), renowned for their interpretability, have become increasingly vital in representing complex stochastic processes in various domains such as gene expression analysis, healthcare, and traffic prediction. Structure learning of DBNs from data is a challenging endeavor, particularly for datasets with thousands of variables. Most current algorithms for DBN structure learning are adaptations from those used in static Bayesian Networks (BNs), and are typically focused on smaller-scale problems. In order to solve large-scale problems while taking full advantage of existing algorithms, this paper introduces a novel divide-and-conquer strategy, originally developed for static BNs, and adapts it for large-scale DBN structure learning. Additionally, we leverage the prior knowledge of 2 Time-sliced BNs (2-TBNs), a special class of DBNs, to enhance the performance of this strategy. Our approach significantly improves the scalability and accuracy of 2-TBN structure learning. Designed experiments demonstrate the effectiveness of our method, showing substantial improvements over existing algorithms in both computational efficiency and structure learning accuracy. In problem instances with more than 1,000 variables, our proposed approach on average improves two accuracy metrics by $74.45\%$ and $110.94\%$, respectively, while reducing runtime by an average of $93.65\%$. Moreover, in problem instances with more than 10,000 variables, our proposed approach successfully completed the task in a matter of hours, whereas the baseline algorithm failed to produce a reasonable result within a one-day runtime limit.