Skip to main content
SpringerLink
Log in

A promotive structural balance model based on reinforcement learning for signed social networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

To solve the structural balance problem in signed social networks, a number of structural balance models have been developed. However, these models neglect the effect of the number of nodes are connected to the changed edges, which is not consistent with the practical requirement of social network systems. For this issue, we propose a novel structural balance model, which jointly takes the minimization of the number of changed edges and the number of nodes connected to the changed edges into account. Then, to optimize the proposed model, we design a novel algorithm based on reinforcement learning, which is a first attempt to use reinforcement learning for structural balance problem. Since nodes in a network don't need to be identified by specific states when solving structural balance problem, a stateless Q-learning is adopted. Furthermore, a policy improvement operator is incorporated into the stateless Q-learning to enhance its ability in exploring solutions in a complex search space. Experimental results on the six networks show that the proposed algorithm has dominance in terms of optimal solutions, stability, and convergence against the other comparison algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. He Q, Wang X, Lei Z, Huang M, Cai Y, Ma L (2019) TIFIM: a two-stage iterative framework for influence maximization in social networks. Appl Math Comput 354:338–352

    MathSciNet  MATH  Google Scholar 

  2. Ma L, Wang XY, Wang XW, Wang L, Shi Y, Huang M (2021) TCDA: truthful combinatorial double auctions for mobile edge computing in industrial internet of things. IEEE Trans Mob Comput. https://doi.org/10.1109/TMC.2021.3064314

    Article  Google Scholar 

  3. Leskovec J, Huttenlocher D, Kleinberg J (2010) Signed networks in social media. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 1361–1370

  4. Chen S, Wang G, Jia W (2015) \(\kappa\)-fuzzytrust: efficient trust computation for large-scale mobile social networks using a fuzzy implicit social graph. Inf Sci 318:123–143

    Article  MathSciNet  Google Scholar 

  5. He Q, Sun L, Wang X, Wang Z, Huang M, Yi B, Wang Y, Ma L (2021) Positive opinion maximization in signed social networks. Inf Sci 558:34–49

    Article  MathSciNet  Google Scholar 

  6. Muchnik L, Aral S, Taylor SJ (2013) Social influence bias: a randomized experiment. Science 341(6146):647–651

    Article  Google Scholar 

  7. Sun Y, Du H, Gong M, Ma LJ, Wang S (2014) Fast computing global structural balance in signed networks based on memetic algorithm. Physica A 415:261–272

    Article  MathSciNet  MATH  Google Scholar 

  8. Facchetti G, Iacono G, Altafini C (2011) Computing global structural balance in large-scale signed social networks. Proc Natl Acad Sci USA 108(52):20953–20958

    Article  Google Scholar 

  9. Wang S, Gong M, Du H, Ma LJ, Miao Q, Du W (2016) Optimizing dynamical changes of structural balance in signed network based on memetic algorithm. Soc Netw 44:64–73

    Article  Google Scholar 

  10. Cai Q, Gong M, Ma L.J, Wang S, Jiao L, Du H (2015) A particle swarm optimization approach for handling network social balance problem. In: IEEE Congress on Evolutionary Computation (CEC). IEEE, pp 3186–3191

  11. Ma LJ, Gong M, Yan J, Yuan F, Du H (2017) A decomposition-based multi-objective optimization for simultaneous balance computation and transformation in signed networks. Inf Sci 378:144–160

    Article  Google Scholar 

  12. Cai Q, Gong M, Ruan S, Miao Q, Du H (2015) Network structural balance based on evolutionary multiobjective optimization: a two-step approach. IEEE Trans Evol Comput 19(6):903–916

    Article  Google Scholar 

  13. Ma L, Huang M, Yang S, Wang R, Wang X (2021) An adaptive localized decision variable analysis approach to large-scale multiobjective and many-objective optimization. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2020.3041212

    Article  Google Scholar 

  14. Ma L, Cheng S, Shi Y (2020) Enhancing learning efficiency of brain storm optimization via orthogonal learning design. IEEE Trans Syst Man Cybern Syst 51:6723–6724. https://doi.org/10.1109/TSMC.2020.2963943

    Article  Google Scholar 

  15. Ali K, Wang C-Y, Yeh M-Y, Chen Y-S (2020) Addressing competitive influence maximization on unknown social network with deep reinforcement learning. In: Proceedings of IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM). IEEE, pp 196–203

  16. Du B, Liu Y, Atiatallah Abbas I (2016) Existence and asymptotic behavior results of periodic solution for discrete-time neutral-type neural networks. J Frankl Inst Eng Appl Math 353(2):448–461. https://doi.org/10.1016/j.jfranklin.2015.11.013

    Article  MathSciNet  MATH  Google Scholar 

  17. He Q, Wang X, Zhao Y, Yi B, Lu X, Yang M, Huang M (2021) Reinforcement-learning-based competitive opinion maximization approach in signed social networks. Trans Comput Soc Syst. https://doi.org/10.1109/TCSS.2021.3120421

    Article  Google Scholar 

  18. Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press, Cambridge

    MATH  Google Scholar 

  19. Antal T, Krapivsky PL, Redner S (2005) Dynamics of social balance on networks. Phys Rev E 72(3):036121

    Article  MathSciNet  Google Scholar 

  20. Marvel SA, Strogatz SH, Kleinberg JM (2009) Energy landscape of social balance. Phys Rev Lett 103(19):198701

    Article  Google Scholar 

  21. Marvel SA, Kleinberg J, Kleinberg RD, Strogatz SH (2011) Continuous-time model of structural balance. Proc Natl Acad Sci USA 108(5):1771–1776

    Article  Google Scholar 

  22. Summers TH, Shames I (2013) Active influence in dynamical models of structural balance in social networks. Europhys Lett 103(1):18001

    Article  Google Scholar 

  23. Deng H, Abell P, Engel O, Wu J, Tan Y (2016) The influence of structural balance and homophily/heterophobia on the adjustment of random complete signed networks. Soc Netw 44:190–201

    Article  Google Scholar 

  24. He X, Du H, Cai M, Feldman MW (2018) The evolution of cooperation in signed networks under the impact of structural balance. PLoS One 13(10):0205084

    Article  Google Scholar 

  25. Altafini C (2012) Dynamics of opinion forming in structurally balanced social networks. PLoS One 7(6):38135

    Article  Google Scholar 

  26. Xia W, Cao M, Johansson KH (2015) Structural balance and opinion separation in trust-mistrust social networks. IEEE Trans Control Netw Syst 3(1):46–56

    Article  MathSciNet  MATH  Google Scholar 

  27. Ma LJ, Huang X, Li J, Lin Q, You Z, Gong M, Leung VC (2019) Privacy-preserving global structural balance computation in signed networks. IEEE Trans Comput Soc Syst 7(1):164–177

    Article  Google Scholar 

  28. Meng D, Du M, Wu Y (2019) Extended structural balance theory and method for cooperative-antagonistic networks. IEEE Trans Autom Control 65(5):2147–2154

    Article  MathSciNet  MATH  Google Scholar 

  29. Heider F (1946) Attitudes and cognitive organization. J Psychol 21(1):107–112

    Article  Google Scholar 

  30. Lerner J (2016) Structural balance in signed networks: separating the probability to interact from the tendency to fight. Soc Netw 45:66–77

    Article  Google Scholar 

  31. Tu Y, Fang H, Yin Y, He S (2021) Reinforcement learning-based nonlinear tracking control system design via LDI approach with application to trolley system. Neural Comput Appl 34:5055–5062. https://doi.org/10.1007/s00521-021-05909-8

    Article  Google Scholar 

  32. He S, Zhang M, Fang H, Liu F, Luan X, Ding Z (2020) Reinforcement learning and adaptive optimization of a class of Markov jump systems with completely unknown dynamic information. Neural Comput Appl 32(2):14311–14320

    Article  Google Scholar 

  33. Su H, Qi W, Yang C, Sandoval J, Ferrigno G, De Momi E (2020) Deep neural network approach in robot tool dynamics identification for bilateral teleoperation. IEEE Robot Autom Lett 5(2):2943–2949

    Article  Google Scholar 

  34. Qi W, Su H, Aliverti A (2020) A smartphone-based adaptive recognition and real-time monitoring system for human activities. IEEE Trans Hum Mach Syst 50(5):414–423. https://doi.org/10.1109/THMS.2020.2984181

    Article  Google Scholar 

  35. Su H, Qi W, Hu Y, Karimi HR, Ferrigno G, Momi ED (2022) An incremental learning framework for human-like redundancy optimization of anthropomorphic manipulators. IEEE Trans Ind Inform 18(3):1864–1872. https://doi.org/10.1109/TII.2020.3036693

    Article  Google Scholar 

  36. Abed-alguni BH (2018) Action-selection method for reinforcement learning based on Cuckoo search algorithm. Arab J Sci Eng 43(12):6771–6785

    Article  Google Scholar 

  37. Abouelmagd EI, Awad M, Elzayat E, Abbas IA (2014) Reduction the secular solution to periodic solution in the generalized restricted three-body problem. Astrophys Space Sci 350(2):495–505

    Article  Google Scholar 

  38. Abed-alguni BH, Ottom MA (2018) Double delayed q-learning. Int J Artif Intell 16(2):41–59

    Google Scholar 

  39. Ma L, Li N, Guo Y, Wang X, Yang S, Huang M, Zhang H (2021) Learning to optimize: reference vector reinforcement learning adaption to constrained many-objective optimization of industrial copper burdening system. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2021.3086501

    Article  Google Scholar 

  40. Abed-Alguni BH, Paul DJ, Chalup SK, Henskens FA (2016) A comparison study of cooperative q-learning algorithms for independent learners. Int J Artif Intell 14(1):71–93

    Google Scholar 

  41. Abed-alguni BH, Chalup SK, Henskens FA, Paul DJ (2015) A multi-agent cooperative reinforcement learning model using a hierarchy of consultants, tutors and workers. Vietnam J Comput Sci 2(4):213–226

    Article  Google Scholar 

  42. Cartwright D, Harary F (1956) Structural balance: a generalization of Heider’s theory. Psychol Rev 63(5):277

    Article  Google Scholar 

  43. Davis JA (1967) Clustering and structural balance in graphs. Hum Relat 20(2):181–187

    Article  Google Scholar 

  44. Liu Y, Liu W, Obaid MA, Abbas IA (2016) Exponential stability of Markovian jumping Cohen–Grossberg neural networks with mixed mode-dependent time-delays. Neurocomputing 177:409–415. https://doi.org/10.1016/j.neucom.2015.11.046

    Article  Google Scholar 

  45. Ma LJ, Gong M, Du H, Shen B, Jiao L (2015) A memetic algorithm for computing and transforming structural balance in signed networks. Knowledge-Based Syst 85:196–209

    Article  Google Scholar 

  46. Albert R, Barabási A-L (2002) Statistical mechanics of complex networks. Rev Mod Phys 74(1):47

    Article  MathSciNet  MATH  Google Scholar 

  47. Watts DJ, Strogatz SH (1998) Collective dynamics of ‘small-world’ networks. Nature 393(6684):440–442

    Article  MATH  Google Scholar 

  48. Kumar S, Hamilton W.L, Leskovec J, Jurafsky D (2018) Community interaction and conflict on the web. In: Proceedings of the 2018 world wide web conference, pp 933–943

  49. Kumar S, Spezzano F, Subrahmanian V, Faloutsos C (2016) Edge weight prediction in weighted signed networks. In: Proceedings of IEEE 16th International Conference on Data Mining (ICDM). IEEE, pp 221–230

  50. Leskovec J, Huttenlocher D, Kleinberg J (2010) Predicting positive and negative links in online social networks. In: Proceedings of the 19th international conference on World wide web. ACM, pp 641–650

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant No. 61872073, No. 62032013, and No. 61773103, the LiaoNing Revitalization Talents Program under Grant No. XLYC1902010, the Fundamental Research Funds for the Central Universities under Grant No. N2117005, and the Joint Funds of the Natural Science Foundation of Liaoning Province under Grant No. 2021-KF-11-01.

Author information

Authors and Affiliations

  1. College of Computer Science and Engineering, Northeastern University, Shenyang, 110169, China

    Mingzhou Yang

  2. State Key Laboratory of Synthetical Automation for Process Industries, College of Computer Science and Engineering, Northeastern University, Shenyang, 110819, China

    Xingwei Wang

  3. State Key Laboratory of Synthetical Automation for Process Industries, College of Software, Northeastern University, Shenyang, 110819, China

    Lianbo Ma

  4. College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, 110169, China

    Qiang He

  5. College of Information Science and Engineering, Northeastern University, Shenyang, 110819, China

    Min Huang

Authors
  1. Mingzhou Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lianbo Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiang He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Min Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xingwei Wang or Lianbo Ma.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, M., Wang, X., Ma, L. et al. A promotive structural balance model based on reinforcement learning for signed social networks. Neural Comput & Applic 34, 16683–16700 (2022). https://doi.org/10.1007/s00521-022-07298-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07298-y

Keywords

Search

Navigation