Skip to main content
SpringerLink
Log in

Scalable Detection of Overlapping Communities and Role Assignments in Networks via Bayesian Probabilistic Generative Affiliation Modeling

  • Conference paper
  • First Online:
Book cover On the Move to Meaningful Internet Systems: OTM 2016 Conferences (OTM 2016)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10033))

Abstract

A new generative model of directed networks is developed to explain link formation from a Bayesian probabilistic perspective. Essentially, nodes can be affiliated to multiple (or, even, all) communities as well as roles. Affiliations are dichotomized to account for link direction. The unknown strength of node affiliations to communities and roles is captured through latent nonnegative random variables, that are ruled by Gamma priors for better model interpretability. Overall, such random variables are meant to generalize both mixed-membership and directed affiliation modeling, which allows for a differentiated connectivity structure inside communities. The probability of a link between two nodes is governed by a Poisson distribution, whose rate increases with the number of shared community affiliations as well as the strength of their affiliations to the common communities and respective roles. The properties of the Poisson distribution are especially beneficial on sparse networks for faster posterior inference. The latter is implemented by a coordinate-ascent variational algorithm enabling affiliation exploration and link prediction. The results of a comparative evaluation carried out on several real-world networks show the overcoming performance of the devised approach in community compactness, link prediction and scalability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Airoldi, E.M., Blei, D.M., Fienberg, S.E., Xing, E.P.: Mixed membership stochastic blockmodels. J. Mach. Learn. Res. 9, 1981–2014 (2008)

    MATH  Google Scholar 

  2. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  3. Blei, D.M., Kucukelbir, A., McAuliffe, J.D.: Variational Inference: A Review for Statisticians. arXiv:1601.00670 (2016)

  4. Chatterjee, N., Sinha, S.: Understanding the mind of a worm: hierarchical network structure underlying nervous system function in c. elegans. In: Banerjee, R., Chakrabarti, B.K. (eds.) Progress in Brain Research, pp. 145–153. Elsevier B.V. (2008)

    Google Scholar 

  5. Chou, B.-H., Suzuki, E.: Discovering community-oriented roles of nodes in a social network. In: Bach Pedersen, T., Mohania, M.K., Tjoa, A.M. (eds.) DaWaK 2010. LNCS, vol. 6263, pp. 52–64. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15105-7_5

    Chapter  Google Scholar 

  6. Costa, G., Ortale, R.: A bayesian hierarchical approach for exploratory analysis of communities and roles in social networks. In: Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pp. 194–201 (2012)

    Google Scholar 

  7. Costa, G., Ortale, R.: Probabilistic analysis of communities and inner roles in networks: Bayesian generative models and approximate inference. Soc. Netw. Anal. Min. 3(4), 1015–1038 (2013)

    Article  Google Scholar 

  8. Costa, G., Ortale, R.: A unified generative bayesian model for community discovery and role assignment based upon latent interaction factors. In: Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pp. 93–100 (2014)

    Google Scholar 

  9. Costa, G., Ortale, R.: Model-based collaborative personalized recommendation on signed social rating networks. ACM Trans. Internet Technol. 16(3), 20:1–20:21 (2016)

    Article  Google Scholar 

  10. Creamer, G., Rowe, R., Hershkop, S., Stolfo, S.J.: Segmentation and automated social hierarchy detection through email network analysis. In: Zhang, H., Spiliopoulou, M., Mobasher, B., Giles, C.L., McCallum, A., Nasraoui, O., Srivastava, J., Yen, J. (eds.) WebKDD/SNA-KDD 2007. LNCS, vol. 5439, pp. 40–58. Springer, Heidelberg (2009). doi:10.1007/978-3-642-00528-2_3

    Chapter  Google Scholar 

  11. Fortunato, S.: Community detection in graphs. Phys. Rep. 486(3–5), 75–174 (2010)

    Article  MathSciNet  Google Scholar 

  12. Girvan, M., Newman, M.E.J.: Community structure in social and biological networks. Proc. Nat. Acad. Sci. 99(12), 7821–7826 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gopalan, P., Hofman, J., Blei, D.: Scalable recommendation with hierarchical poisson factorization. In: Proceedings of Uncertainty in Artificial Intelligence, pp. 326–335 (2015)

    Google Scholar 

  14. Gopalan, P., Wang, C., Blei, D.M.: Modeling overlapping communities with node popularities. In: Proceedings of Advances in Neural Information Processing Systems, pp. 2850–2858 (2013)

    Google Scholar 

  15. Gopalan, P.K., Blei, D.M.: Efficient discovery of overlapping communities in massive networks. Proc. Natl. Acad. Sci. 110(36), 14534–14539 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  16. Henderson, K., Eliassi-Rad, T., Papadimitriou, S., Faloutsos, C.: Hcdf: a hybrid community discovery framework. In: SIAM SDM, pp. 754–765 (2010)

    Google Scholar 

  17. Henderson, K., Eliassi Rad, T.: Applying latent dirichlet allocation to group discovery in large graphs. In: ACM SAC, pp. 1456–1461 (2009)

    Google Scholar 

  18. Kernighan, B.W., Lin, S.: An efficient heuristic procedure for partitioning graphs. Bell Syst. Tech. J. 49(1), 291–307 (1970)

    Article  MATH  Google Scholar 

  19. Koller, D., Friedman, N.: Probabilistic Graphical Models. Principles and Techniques. The MIT Press, Cambridge (2009)

    MATH  Google Scholar 

  20. Liben-Nowell, D., Kleinberg, J.: The link-prediction problem for social networks. J. Am. Soc. Inform. Sci. Technol. 58(7), 1019–1031 (2007)

    Article  Google Scholar 

  21. Lorrain, F., White, H.C.: The structural equivalence of individuals in social networks. J. Math. Sociol. 1, 49–80 (1971)

    Article  Google Scholar 

  22. McAuley, J., Leskovec, J.: Learning to discover social circles in ego networks. In: Proceedings of Advances in Neural Information Processing Systems, pp. 548–556 (2012)

    Google Scholar 

  23. McCallum, A., Wang, X., Corrada-Emmanuel, A.: Topic and role discovery in social networks with experiments on enron and academic email. J. Artif. Intell. Res. 30(1), 249–272 (2007)

    Google Scholar 

  24. Newman, M.E.J.: Detecting community structure in networks. Eur. Phys. J. B 38(2), 321–330 (2004)

    Article  Google Scholar 

  25. Newman, M.E.J.: Fast algorithm for detecting community structure in networks. Phys. Rev. E 69, 066133 (2004)

    Article  Google Scholar 

  26. Newman, M.E.J., Girvan, M.: Finding and evaluating community structure in networks. Phys. Rev. E 69(2), 026113 (2004)

    Article  Google Scholar 

  27. Pathak, N., Delong, C., Banerjee, A., Erickson, K.: Social topic models for community extraction. In: Proceedings of KDD Workshop on Social Network Mining and Analysis (2008)

    Google Scholar 

  28. Pothen, A., Simon, H.D., Liou, K.-P.: Partitioning sparse matrices with eigenvectors of graphs. SIAM J. Matrix Anal. Appl. 11(3), 430–452 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  29. Radicchi, F., Castellano, C., Cecconi, F., Loreto, V., Parisi, D.: Defining and identifying communities in networks. In: Proceedings of the National Academy of Sciences of the United States of America, vol. 101(9), pp. 2658–2663 (2004)

    Google Scholar 

  30. Scripps, J., Tan, P.-N., Esfahanian, A.-H.: Exploration of link structure and community-based node roles in network analysis. In: Proceeding of International Conference on Data Mining, pp. 649–654 (2007)

    Google Scholar 

  31. Scripps, J., Tan, P.-N., Esfahanian, A.-H.: Node roles and community structure in networks. In: Proceedings of Workshop on Web Mining and Social Network Analysis (WebKDD and SNA-KDD), pp. 26–35 (2007)

    Google Scholar 

  32. Sohn, Y., Choi, M.-K., Ahn, Y.-Y., Lee, J., Jeong, J.: Topological cluster analysis reveals the systemic organization of the caenorhabditis elegans connectome. PLoS Comput. Biol. 7(5), e1001139 (2011)

    Article  MathSciNet  Google Scholar 

  33. Wasserman, S., Faust, K.: Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge (1994)

    Book  MATH  Google Scholar 

  34. Watts, D.J., Strogatz, S.H.: Collective dynamics of small-world networks. Nature 393(6684), 440–442 (1998)

    Article  Google Scholar 

  35. White, J.G., Southgate, E., Thompson, J.N., Brenner, S.: The structure of the nervous system of the nematode caenorhabditis elegans. Philos. Trans. R. Soc. B: Biol. Sci. 314(1165), 1–340 (1986)

    Article  Google Scholar 

  36. Xie, J., Kelley, S., Szymanski, B.K.: Overlapping community detection in networks: the state of the art and comparative study. ACM Comput. Surv. 45(4), 43:1–43:35 (2013)

    Article  MATH  Google Scholar 

  37. Yang, J., Leskovec, J.: Structure and overlaps of ground-truth communities in networks. ACM Trans. Intell. Syst. Technol. 5(2), 26:1–26:35 (2014)

    Article  Google Scholar 

  38. Yang, J., McAuley, J., Leskovec, J.: Community detection in networks with node attributes. In: Proceedings of International Conference on Data Mining, pp. 1151–1156 (2013)

    Google Scholar 

  39. Yang, J., McAuley, J., Leskovec, J.: Detecting cohesive and 2-mode communities in directed and undirected networks. In: Proceedings of ACM International Conference on Web Search and Data Mining, pp. 323–332 (2014)

    Google Scholar 

  40. Zhang, H., Qiu, B., Giles, C.L., Foley, H.C., Yen, J.: An lda-based community structure discovery approach for large-scale social networks. In: Proceedings of IEEE International Conference on Intelligence and Security Informatics, pp. 200–207 (2007)

    Google Scholar 

  41. Zhou, D., Manavoglu, E., Li, J., Giles, C.L., Zha, H.: Probabilistic models for discovering e-communities. In: Proceedings of International Conference on World Wide Web, pp. 173–182 (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ICAR-CNR, Via Bucci 41c, 87036, Rende (CS), Italy

    Gianni Costa & Riccardo Ortale

Authors
  1. Gianni Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riccardo Ortale
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Riccardo Ortale .

Editor information

Editors and Affiliations

  1. ADAPT Centre, Trinity College Dublin, Dublin 2, Ireland

    Christophe Debruyne

  2. University of Lorraine, Vandoeuvre-les-Nancy, France

    Hervé Panetto

  3. TU Graz, Graz, Austria

    Robert Meersman

  4. La Trobe University, Melbourne, Australia

    Tharam Dillon

  5. Institute of Computer Languages, TU Wien, Vienna, Austria

    eva Kühn

  6. ADAPT Centre, Trinity College Dublin, Dublin 2, Ireland

    Declan O'Sullivan

  7. Università degli Studi di Milano Crema, Crema, Italy

    Claudio Agostino Ardagna

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Costa, G., Ortale, R. (2016). Scalable Detection of Overlapping Communities and Role Assignments in Networks via Bayesian Probabilistic Generative Affiliation Modeling. In: Debruyne, C., et al. On the Move to Meaningful Internet Systems: OTM 2016 Conferences. OTM 2016. Lecture Notes in Computer Science(), vol 10033. Springer, Cham. https://doi.org/10.1007/978-3-319-48472-3_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-48472-3_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-48471-6

  • Online ISBN: 978-3-319-48472-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation