Skip to main content
SpringerLink
Log in

An Efficient Data Distribution Strategy for Distributed Graph Processing System

  • Conference paper
  • First Online:
Computer Information Systems and Industrial Management (CISIM 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13293))

Abstract

Big data applications like social networks, biological networks, etc. are often realized on graphs. Graph processing, if done on a single node, increases time complexity. Partitioning of graphs has been proved to be useful towards handle this well-known issue. There are several partitioning algorithms that are used to partition a graph. Each partition is assigned to a node within a cluster. However, the storage capacity of a node is limited. Therefore, an effective data distribution mechanism is required. This work aims to propose a novel strategy that would define an efficient distribution of graphs into nodes using genetic algorithms. The proposed data distribution strategy, when applied on two benchmark data set, shows improved data availability without increasing the number of replicas. It has also observed that the execution time will almost became half after applying the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akbari, M., Rashidi, H., Alizadeh, S.H.: An enhanced genetic algorithm with new operators for task scheduling in heterogeneous computing systems. Eng. Appl. Artif. Intell. 61, 35–46 (2017)

    Article  Google Scholar 

  2. Alekseev, V.E., Boliac, R., Korobitsyn, D.V., Lozin, V.V.: NP-hard graph problems and boundary classes of graphs. Theoret. Comput. Sci. 389(1–2), 219–236 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  3. Bentley, J.L.: Multidimensional divide-and-conquer. Commun. ACM 23(4), 214–229 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  4. Cameron, K., Eschen, E.M., Hoàng, C.T., Sritharan, R.: The complexity of the list partition problem for graphs. SIAM J. Discret. Math. 21(4), 900–929 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  5. Day, W.H., Edelsbrunner, H.: Efficient algorithms for agglomerative hierarchical clustering methods. J. Classif. 1(1), 7–24 (1984)

    Article  MATH  Google Scholar 

  6. Golab, L., Hadjieleftheriou, M., Karloff, H., Saha, B.: Distributed data placement to minimize communication costs via graph partitioning. In: Proceedings of the 26th International Conference on Scientific and Statistical Database Management, pp. 1–12 (2014)

    Google Scholar 

  7. Gowda, K.C., Krishna, G.: Agglomerative clustering using the concept of mutual nearest neighbourhood. Pattern Recogn. 10(2), 105–112 (1978)

    Article  MATH  Google Scholar 

  8. Kalavri, V., Vlassov, V., Haridi, S.: High-level programming abstractions for distributed graph processing. IEEE Trans. Knowl. Data Eng. 30(2), 305–324 (2017)

    Article  Google Scholar 

  9. Leskovec, J., Mcauley, J.: Learning to discover social circles in ego networks. Adv. Neural Inf. Process. Syst. 25 (2012)

    Google Scholar 

  10. Lu, W., Shen, Y., Wang, T., Zhang, M., Jagadish, H.V., Du, X.: Fast failure recovery in vertex-centric distributed graph processing systems. IEEE Trans. Knowl. Data Eng. 31(4), 733–746 (2018)

    Article  Google Scholar 

  11. Margo, D., Seltzer, M.: A scalable distributed graph partitioner. Proc. VLDB Endow. 8(12), 1478–1489 (2015)

    Article  Google Scholar 

  12. Murtagh, F., Contreras, P.: Algorithms for hierarchical clustering: an overview, II. Wiley Interdisc. Rev.: Data Min. Knowl. Discov. 7(6), e1219 (2017)

    Google Scholar 

  13. Paranjape, A., Benson, A.R., Leskovec, J.: Motifs in temporal networks. In: Proceedings of the tenth ACM International Conference on Web Search and Data Mining, pp. 601–610 (2017)

    Google Scholar 

  14. Phan, T., Ranganathan, K., Sion, R.: Evolving toward the perfect schedule: co-scheduling job assignments and data replication in wide-area systems using a genetic algorithm. In: Feitelson, D., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 173–193. Springer, Heidelberg (2005). https://doi.org/10.1007/11605300_9

    Chapter  Google Scholar 

  15. Prakash, S., Vidyarthi, D.P.: Maximizing availability for task scheduling in computational grid using genetic algorithm. Concurr. Comput.: Pract. Exp. 27(1), 193–210 (2015)

    Article  Google Scholar 

  16. Rahimian, F., Payberah, A.H., Girdzijauskas, S., Jelasity, M., Haridi, S.: A distributed algorithm for large-scale graph partitioning. ACM Trans. Auton. Adapt. Syst. (TAAS) 10(2), 1–24 (2015)

    Article  Google Scholar 

  17. Sajjad, H.P., Rahimian, F., Vlassov, V.: Smart partitioning of geo-distributed resources to improve cloud network performance. In: 2015 IEEE 4th International Conference on Cloud Networking (CloudNet), pp. 112–118. IEEE (2015)

    Google Scholar 

  18. Shahapure, K.R., Nicholas, C.: Cluster quality analysis using silhouette score. In: 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA), pp. 747–748. IEEE (2020)

    Google Scholar 

  19. Sun, J., Dong, X., Zhang, X., Wang, Y.: An availability approached task scheduling algorithm in heterogeneous fault-tolerant system. In: 2014 9th IEEE International Conference on Networking, Architecture, and Storage, pp. 275–280. IEEE (2014)

    Google Scholar 

  20. Whitley, D.: A genetic algorithm tutorial. Stat. Comput. 4(2), 65–85 (1994)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Calcutta JD - II, Sector III, Salt Lake City, Kolkata, 700106, India

    Aradhita Mukherjee & Nabendu Chaki

  2. A. K. Choudhury School of Information Technology, University of Calcutta JD - II, Sector III, Salt Lake City, Kolkata, 700106, India

    Rituparna Chaki

Authors
  1. Aradhita Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rituparna Chaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nabendu Chaki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aradhita Mukherjee .

Editor information

Editors and Affiliations

  1. Bialystok University of Technology, Bialystok, Poland

    Khalid Saeed

  2. VSB - Technical University of Ostrava, Ostrava, Czech Republic

    Jiří Dvorský

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mukherjee, A., Chaki, R., Chaki, N. (2022). An Efficient Data Distribution Strategy for Distributed Graph Processing System. In: Saeed, K., Dvorský, J. (eds) Computer Information Systems and Industrial Management. CISIM 2022. Lecture Notes in Computer Science, vol 13293. Springer, Cham. https://doi.org/10.1007/978-3-031-10539-5_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-10539-5_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-10538-8

  • Online ISBN: 978-3-031-10539-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation