Skip to main content
SpringerLink
Log in

Data Parallel Implementation of Belief Propagation in Factor Graphs on Multi-core Platforms

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

We investigate data parallel techniques for belief propagation in acyclic factor graphs on multi-core systems. Belief propagation is a key inference algorithm in factor graph, a probabilistic graphical model that has found applications in many domains. In this paper, we explore data parallelism for basic operations over the potential tables in belief propagation. Data parallel techniques for these table operations are developed for shared memory platforms. We then propose a complete belief propagation algorithm using these table operations to perform exact inference in factor graphs. The proposed algorithms are implemented on state-of-the-art multi-socket multi-core systems with additional NUMA-aware optimizations. Our proposed algorithms exhibit good scalability using a representative set of factor graphs. On a four-socket Intel Westmere-EX system with 40 cores, we achieve 39.5\(\times \) speedup for the table operations and 39\(\times \) speedup for the complete algorithm using factor graphs with large potential tables.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Agarwal, V., Petrini, F., Pasetto, D., Bader, D.: Scalable graph exploration on multicore processors. In: Proceedings of the 22nd IEEE and ACM Supercomputing Conference (SC10) (2010)

  2. Bader, D.A., JáJá, J., Chellappa, R.: Scalable data parallel algorithms for texture synthesis and compression using Gibbs random fields. IEEE Trans. Image Process. 4(10), 1456–1460 (1995)

    Article  Google Scholar 

  3. Buyya, R.: High Performance Cluster Computing: Architectures and Systems, Vol. 1. Prentice Hall, Englewood Cliffs (1999)

  4. Coarfa, C., Dotsenko, Y., Mellor-Crummey, J., Cantonnet, F., El-Ghazawi, T., Mohanti, A., Yao, Y., Chavarría-Miranda, D.: An evaluation of global address space languages: co-array Fortran and unified parallel C. In: Proceedings of the Tenth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP ’05, pp. 36–47. ACM, New York (2005)

  5. Falcão, G., Sousa, L., Silva, V.: Massively LDPC decoding on multicore architectures. IEEE Trans. Parallel Distrib. Syst. 22(2), 309–322 (2011)

    Article  Google Scholar 

  6. Gonzalez, J., Low, Y., Guestrin, C., O’Hallaron, D.: Distributed parallel inference on large factor graphs. In: Conference on Uncertainty in Artificial Intelligence (UAI). Montreal, Canada (2009)

  7. JáJá, J.: An Introduction to Parallel Algorithms. Addison-Wesley, Reading (1992)

    MATH  Google Scholar 

  8. Karkooti, M., Cavallaro, J.: Semi-parallel reconfigurable architectures for real-time LDPC decoding. In: Proceedings of International Conference on Information Technology: Coding and, Computing (2004)

  9. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. The MIT Press, Cambridge (2009)

  10. Kozlov, A.V., Singh, J.P.: A parallel Lauritzen-Spiegelhalter algorithm for probabilistic inference. In: Proceedings of Supercomputing, pp. 320–329 (1994)

  11. Kschischang, F.R., Frey, B.J., Loeliger, H.A.: Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory 47(2), 498–519 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  12. Kurzak, J., Bader, D.A., Dongarra, J.: Scientific Computing with Multicore and Accelerators. Chapman & Hall/CRC Press, London (2010)

    Book  Google Scholar 

  13. Lauritzen, S.L., Spiegelhalter, D.J.: Local computation with probabilities and graphical structures and their application to expert systems. R. Stat. Soc. B 50, 157–224 (1988)

    MATH  MathSciNet  Google Scholar 

  14. Loeliger, H.: An introduction to factor graphs. IEEE Signal Process. Mag. 21(1), 28–41 (2004)

    Article  Google Scholar 

  15. Mendiburu, A., Santana, R., Lozano, J.A., Bengoetxea, E.: A parallel framework for loopy belief propagation. In: GECCO (Companion), pp. 2843–2850 (2007)

  16. Mitrofanova, A., Pavlovic, V., Mishra, B.: Integrative protein function transfer using factor graphs and heterogeneous data sources. In: Proceedings of the 2008 IEEE International Conference on Bioinformatics and Biomedicine, pp. 314–318. IEEE Computer Society (2008)

  17. Mooij, J.M.: libDAI: a free and open source C++ library for discrete approximate inference in graphical models. J. Mach. Learn. Res. 11, 2169–2173 (2010). http://www.jmlr.org/papers/volume11/mooij10a/mooij10a.pdf

  18. Parashar, M., Li, X.: Advanced Computational Infrastructures for Parallel and Distributed Applications, Chap. Enabling Large-Scale Computational Science: Motivations, Requirements and, Challenges, pp. 1–7 (2010)

  19. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)

    Google Scholar 

  20. Piatkowski, N., Morik, K.: Parallel inference on structured data with CRFs on GPUs. In: International Workshop at ECML PKDD on Collective Learning and Inference on Structured Data (COLISD2011) (2011)

  21. Richardson, T., Urbanke, R.: The capacity of low-density parity check codes under message-passing decoding. IEEE Trans. Inf. Theory 47(2), 599–618 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  22. Schlegel, C., Perez, L.: Trellis and Turbo Coding. Wiley, New York (2003)

    Google Scholar 

  23. Singh, S., Subramanya, A., Pereira, F., McCallum, A.: Distributed map inference for undirected graphical models. In: Neural Information Processing Systems (NIPS), Workshop on Learning on Cores, Clusters and Clouds (2010)

  24. Spafford, K., Meredith, J., Vetter, J.: Maestro: data orchestration and tuning for OpenCL devices. In: Proceedings of the 16th international Euro-Par conference on Parallel processing: Part II, Euro-Par’10, pp. 275–286. Springer, Berlin (2010)

  25. Sudderth, E., Freeman, W.T.: Signal and image processing with belief propagation. IEEE Signal Process. Mag. 25, 114–141 (2008)

    Google Scholar 

  26. Verghese, B., Devine, S., Gupta, A., Rosenblum, M.: Operating system support for improving data locality on CC-NUMA compute servers. In: Proceedings of the Seventh International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS-VII, pp. 279–289. ACM (1996)

  27. Xia, Y., Prasanna, V.K.: Scalable node level computation kernels for parallel exact inference. IEEE Trans. Comput. 59(1), 103–115 (2009)

    Article  MathSciNet  Google Scholar 

  28. Xia, Y., Prasanna, V.K.: Parallel evidence propagation on multicore processors. J. Supercomput. 57, 189–202 (2010)

    Article  Google Scholar 

Download references

Acknowledgments

This research was partially supported by the U.S. National Science Foundation under grant number CNS-1018801. Support from the Intel Manycore Testing Lab is gratefully acknowledged.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Southern California, Los Angeles, CA, 90089, USA

    Nam Ma

  2. IBM T.J. Watson Research Center, Yorktown Heights, NY, 10598, USA

    Yinglong Xia

  3. Ming Hsieh Department of Electrical Engineering, University of Southern California, Los Angeles, CA, 90089, USA

    Viktor K. Prasanna

Authors
  1. Nam Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinglong Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Viktor K. Prasanna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nam Ma.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ma, N., Xia, Y. & Prasanna, V.K. Data Parallel Implementation of Belief Propagation in Factor Graphs on Multi-core Platforms. Int J Parallel Prog 42, 219–237 (2014). https://doi.org/10.1007/s10766-013-0246-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-013-0246-9

Keywords

Search

Navigation