Abstract
Belief propagation (BP) is a message passing algorithm that infers over probabilistic graphical models. Its main computational workload, messages update, is suitable for GPU’s massively parallel architecture. However, the efficiency of fully parallel BP is low, and traditional algorithms implemented on GPUs occupy amount of computing and memory resources. In this paper, we propose several GPU-friendly BP algorithms optimized by coloring. Color Wave (CW) algorithm performs multi-step coloring on residuals of non-convergent vertices to quickly obtain multiple disjoint partitions and vertices with the largest residuals in each partition, and then updates batches of messages in a fixed order. These operations are all suitable for parallelization and require little additional memory. To save time in each iteration, the Color Extract (CE) algorithm only update messages on edges with the largest residuals among all adjacent edges. The Random Drop (RD) algorithm steadily increases the convergence degree by progressively reducing the messages update ratio of non-convergent edges. The experiments on different GPUs show that our algorithms perform well throughout the calculation process. Compared with state-of-the-art algorithms, CW algorithm converged most of the messages in previous iterations. The convergence degree of CE is higher than all other algorithms in most calculation processes. RD converges fast and always has a high degree of convergence.
This work was supported by the National Natural Science Foundation of China (No. 61931019).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, Z., Liu, R., Yan, Z., Zhao, L.: GPU-based implementation of belief propagation decoding for polar codes. In: International Conference on Acoustics Speech and Signal Processing (ICASSP 2019), pp. 1513–1517. IEEE, Brighton (2019)
Shan, B., Fang, Y.: GPU accelerated parallel algorithm of sliding-window belief propagation for LDPC codes. Int. J. Parallel Prog. 48(3), 566–579 (2020)
Romero, D.L., Chang, N.B.: Sequential decoding of non-binary LDPC codes on graphics processing units. In: Asilomar Conference on Signals, Systems and Computers (ACSCC 2012), pp. 1267–1271. IEEE, Pacific Grove (2012)
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2004), pp. 261–268. IEEE, Washington (2004)
Yanover, C., Weiss, Y.: Approximate inference and protein-folding. In: Neural Information Processing Systems (NIPS 2002), pp. 1481–1488. Vancouver (2002)
Wang, H., Geng, L., Lee, R., Hou, K., Zhang, Y., Zhang, X.: SEP-graph: finding shortest execution paths for graph processing under a hybrid framework on GPU. In: ACM Sigplan Symposium on Principles and Practice of Parallel Programming (PPoPP 2019), pp. 38–52. ACM, Washington (2019)
Segura, A., Arnau, J.-M., González, A.: SCU: a GPU stream compaction unit for graph processing. In: International Symposium on Computer Architecture (ISCA 2019), pp. 424–435. IEEE, Phoenix (2019)
Alabandi, G., Powers, E., Burtscher, M.: Increasing the parallelism of graph coloring via shortcutting. In: ACM Sigplan Symposium on Principles and Practice of Parallel Programming (PPoPP 2020), pp. 262–275. ACM, San Diego (2020)
Elidan, G., McGraw, I., Koller, D.: Residual belief Propagation: informed scheduling for asynchronous message passing. In: Uncertainty in Artificial Intelligence (UAI 2006), Cambridge, pp. 165–173 (2006)
Murphy, K., Weiss, Y., Jordan, M.I.: Loopy belief propagation for approximate inference: an empirical study. In: Uncertainty in Artificial Intelligence (UAI 1999), Stockholm, pp. 467–475 (1999)
Gonzalez, J., Low, Y., Guestrin, C.: Residual splash for optimally parallelizing belief propagation. In: International Conference on Artificial Intelligence and Statistics (AISTATS 2009), Clearwater Beach, pp. 177–184 (2009)
Der Merwe, M.V., Joseph, V., Gopalakrishnan, G.: Message scheduling for performant, many-core belief propagation. In: IEEE High Performance Extreme Computing Conference (HPEC 2019), pp. 1–7. IEEE, Waltham (2019)
Mooij, J.M., Kappen, H.J.: Sufficient conditions for convergence of the sum-product algorithm. IEEE Trans. Inf. Theory 53(12), 4422–4437 (2007)
Wang, Y., et al.: Gunrock: GPU graph analytics. ACM Trans. Parallel Comput. 4(1), 1–49 (2017)
Pingali, K., et al.: The tao of parallelism in algorithms. In: SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2011), pp. 12–25. ACM, San Jose (2011)
Szeliski, R., et al.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1068–1080 (2008)
Choi, J., Patil, A.D., Rutenbar, R.A., Shanbhag, N.R.: Analysis of error resiliency of belief propagation in computer vision. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2016), pp. 1060–1064. IEEE, Shanghai (2016)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Hou, J., Si, C., Wang, S., Wu, G., Zhang, L. (2020). Parallel Belief Propagation Optimized by Coloring on GPUs. In: Qiu, M. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2020. Lecture Notes in Computer Science(), vol 12452. Springer, Cham. https://doi.org/10.1007/978-3-030-60245-1_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-60245-1_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60244-4
Online ISBN: 978-3-030-60245-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)