Skip to main content
SpringerLink
Log in

D-HOCS: an algorithm for discovering the hierarchical overlapping community structure of a social network

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Social networks often demonstrate a hierarchical organization, with communities embedded within other communities; moreover, nodes can be shared between different communities, i.e. communities in social networks may be overlapping. In this paper, we define a hierarchical overlapping community structure to present overlapping communities of a social network at different levels of granularity. Discovering the hierarchical overlapping community structure of a social network can provide us a deeper understanding of the complex nature of social networks. We propose an algorithm, called D-HOCS, to derive the hierarchical overlapping community structure of social networks. Firstly, D-HOCS generates a probability transition matrix by applying random walk to a social network, and then trains a Gaussian Mixture Model using the matrix. Further D-HOCS derives overlapping communities by analyzing mean vectors of the Gaussian mixture model. Varying the number of components, D-HOCS repeatedly trains the Gaussian mixture model, detecting the overlapping communities at different levels of granularity. Organizing the overlapping communities into a hierarchy, D-HOCS can finally obtain the hierarchical overlapping community structure of the social network. The experiments conducted on synthetic and real dataset demonstrate the feasibility and applicability of the proposed algorithm. We further employ D-HOCS to explore Enron e-mail corpus, and obtain several interesting insights. For example, we find out a coordinator who coordinated many sections of the Enron Corporation to complete an important task during first half of 2001. We also identify a community that corresponds to a real organization in Enron Corporation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://www.isi.edu/~adibi/Enron/Enron_Employee_Status.xls

  2. http://en.wikipedia.org/wiki/Enron_scandal#Timeline_of_downfall

References

  • Ahn, Y.-Y., Bagrow, J.P., Lehmann, S. (2010). Link communities reveal multiscale complexity in networks. Nature, 466(5), 761–764.

    Article  Google Scholar 

  • Chapanond, A., Krishnamoorthy, M.S., Yener, B. (2005). Graph theoretic and spectral analysis of Enron e-mail data. Computational & Mathematical Organization Theory, 11, 261–281.

    Article  Google Scholar 

  • Chekuri, C., Goldberg, A., Karger, D., Levin, M., Stein, C. (1997). Experimentao study of minimum cut algorithms. In Proc. 8th ACM-SAIM symposium on discreet algorithm (pp. 324–333).

  • Clauset, A., Moore, C., Newman, M.E.J. (2008). Hierarchical structure and the prediction of missing links in networks. Nature, 453, 98–101.

    Article  Google Scholar 

  • Craswell, N., & Szummer, M. (2007). Random walks on the click graph. In Proceedings of the 30th annual international ACM SIGIR conference (pp. 239–246).

  • Diesner, J., Frantz, T.L., Carley, K.M. (2005). Communication network from the enrom e-mail corpus “It’s always about the people. Enron is no different”. Computational & Mathematical Organization Theory, 11, 201–228.

    Article  MATH  Google Scholar 

  • Ding, C., He, X., Zha, H., Gu, M., Simon, H. (2001). A min-max cut algorithm for graph partitioning and data clustering. In Proc. of ICDM 2001 (pp. 107–114).

  • Friggeri, A., Chelius, G., Fleury, E. (2011). Triangles to capture social cohesion. In 2011 IEEE international conference on privacy, security, risk, and trust, and IEEE international conference on social computing (pp. 258–265).

  • Gong, M., Chen, X., Ma, L., Zhang, Q., Jiao, L. (2013). Identification of multi-resolution network structures with multi-objective immune algorithm. Applied Soft Computing, 13, 1705–1717.

    Article  Google Scholar 

  • Gregory, S. (2009). Finding overlapping communities using disjoint community detection algorithms. In Complex Networks, Studies in Computational Intelligence (vol. 207, pp. 47–61). Springer, Heidelberg.

    Google Scholar 

  • Gregory, S. (2011). Fuzzy overlapping communities in networks. Journal of Statistical Mechanics Theory and Experiment, 2011(2), P02017.

    Article  Google Scholar 

  • Huang, J., Sun, H., Han, J., Deng, H., Sun, Y., Liu, Y. (2010). SHRINK: a structural clustering algorithm for detecting hierarchical communities in networks. In Proceedings of CIKM’10 (pp. 219–228).

  • Lancichinetti, A., Fortunato, S., Kertész, J. (2009). Detecting the overlapping and hierarchical community structure in complex networks. New Journal of Physics, 11(3), 033015.

    Article  Google Scholar 

  • Lee, C., Reid, F., McDaid, A., Hurley, N. (2010). Detecting highly overlapping community structure by greedy clique expansion. In 4th SNA-KDD Workshop ’10 (SNA-KDD’10) (pp. 112–119).

  • Leicht, E.A., Clarkson, G., Shedden, K., Newman, M.E.J. (2007). Large-scale structure of time evolving citation networks. The European Physical Journal B; Condensed Matter and Complex Systems, 59(1), 75–83.

    Article  MATH  Google Scholar 

  • McCallum, A., & Wang, X. (2007). Topic and role discovery in social networks with experiments on Enron and academic e-mail. Journal of Artificial Intelligence Research, 30, 249–272.

    Google Scholar 

  • Newman, M.E.J. (2004). Fast algorithm for detecting community structure in networks. Physical Review, E69, 066133.

    Google Scholar 

  • Newman, M.E.J. (2010). Networks: An introduction. Oxford, UK: Oxford University Press.

    Book  Google Scholar 

  • Newman, M.E.J., & Girvan, M. (2004). Finding and evaluating community structure in networks. Physical Review, E69(026113), 1–15.

    Google Scholar 

  • Palla, G., Derényi, I., Farkas, I., Vicsek, T. (2005). Uncovering the overlapping community structure of complex networks in nature and society. Nature, 435, 814–818.

    Article  Google Scholar 

  • Shen, H., Cheng, X., Cai, K., Hu, M.-B. (2009). Detect overlapping and hierarchical community structure in networks. Physica A Statistical Mechanics and its Applications, 388(8), 1706–1712.

    Article  Google Scholar 

  • Shen, H.-W., Cheng, X.-Q., Guo, J.-F. (2009). Quantifying and identifying the overlapping community structure in networks. Journal of Statistical Mechanics: Theory and Experiment, 2009(7), P07042.

    Article  Google Scholar 

  • Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.

    Article  Google Scholar 

  • Xu, X., Yuruk, N., Feng, Z., Schweiger, T.A.J. (2007). SCAN: a structural clustering algorithm for networks. In Proceeding of SIGKDD’07 (pp 824–833).

  • Yang, B., Jin, D., Liu, J., Liu, D. (2013). Hierarchical community detection with applications to real-world network analysis. Data & Knowledge Engineering, 83, 20–38.

    Article  Google Scholar 

  • Zachary, W.W. (1977). An information flow model for conflict and fission in small groups. Journal of Anthropological Research, 33, 452–473.

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Major Program of the National Natural Science Foundation of China (Grant Number: 91218301), as well as the Fundamental Research Funds for the Central University of China (JBK130923, JBK120515).

Author information

Authors and Affiliations

  1. School of Information, Southwestern University of Finance and Economics, Chengdu, China

    Jiangtao Qiu

  2. The Rawls College of Business Administration, Texas Tech University, Lubbock, TX, USA

    Zhangxi Lin

Authors
  1. Jiangtao Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhangxi Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiangtao Qiu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Qiu, J., Lin, Z. D-HOCS: an algorithm for discovering the hierarchical overlapping community structure of a social network. J Intell Inf Syst 42, 353–370 (2014). https://doi.org/10.1007/s10844-013-0272-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-013-0272-5

Keywords

Search

Navigation