Skip to main content
SpringerLink
Log in

A Self-Organized Grouping (SOG) Framework for Efficient Grid Resource Discovery

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Dynamic and heterogeneous characteristics of large-scale Grids make the fundamental problem of resource discovery a great challenge. This paper presents a self-organized grouping (SOG) framework that achieves efficient Grid resource discovery by forming and maintaining autonomous resource groups. Each group dynamically aggregates a set of resources together with respect to similarity metrics of resource characteristics. The SOG framework takes advantage of the strengths of both centralized and decentralized approaches that were previously developed for Grid/P2P resource discovery. The design of SOG minimizes the overhead incurred by the process of group formation and maximizes the performance of resource discovery. The way SOG approach handles resource discovery queries is metaphorically similar to searching for a word in an English dictionary, by identifying its alphabetical group at the first place, and then performing a lexical search within the group. Because multi-attribute range queries represent an important aspect of resource discovery, we devise a generalized approach using a space-filling curve in conjunction with the SOG framework. We exploit the Hilbert space-filling curve’s locality preserving and dimension reducing mapping. This mapping provides a 1-dimensional grouping attribute to be used by the SOG framework. Experiments show that the SOG framework achieves superior look-up performance that is more scalable, stable and efficient than other existing approaches. Furthermore, our experimental results indicate that the SOG framework has little dependence on factors such as resource density, query type, and Grid size.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Andrzejak, A., Xu, Z.: Scalable, efficient range queries for Grid information services. In: P2P ’02: Proceedings of the Second International Conference on Peer-to-Peer Computing, p. 33. IEEE Computer Society, Washington, DC (2002)

    Chapter  Google Scholar 

  2. Asano, T., Ranjan, D., Roos, T., Welzl, E., Widmayer, P.: Space-filling curves and their use in the design of geometric data structures. Theor. Comp. Sci. 181(1), 3–15 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  3. Ayalvadi, G., Kermarrec, A., Massoulie, L., SCAMP,: peer-to-peer lightweight membership service for large-scale group communication. In: Proc. 3rd Intnl. Wshop Networked Group Communication (NGC ’01), LNCS 2233, pp. 44–55. Springer, New York (2001)

    Google Scholar 

  4. Bharambe, A.R., Agrawal, M., Seshan, S.: Mercury: supporting scalable Multi-attribute range queries. In: SIGCOMM ’04: proceedings of the 2004 conference on applications, technologies, architectures, and protocols for computer communications, pp. 353–366 (2004)

  5. Butz, A.R.: Alternative algorithm for Hilbert’s space-filling curve. IEEE Trans. Comp. C-20, 424–426 (1971)

    Article  Google Scholar 

  6. Caron, E., Desprez, F., Tedeschi, C.: Enhancing computational Grids with peer-to-peer technology for large scale service discovery. Journal of Grid. Computing 5(3), 337–360 (2007)

    Article  Google Scholar 

  7. Chen, G., Ping Low, C., Yang, Z.: Enhancing search performance in unstructured P2P networks based on users’ common interest. IEEE Trans. Parallel Distrib. Syst. 19(6), 821–836 (2008)

    Article  Google Scholar 

  8. Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid information services for distributed resource sharing. In: Proceedings of the Tenth IEEE International Symposium on High-Performance Distributed Computing (HPDC-10) (2001)

  9. Eva, S., Manohar, R., Chandy, M.: An analysis of leader election for multicast groups. Technical report, AT&T Labs-Research (2002)

  10. Forestiero, A., Mastroianni, C., Spezzano, G.: Building a peer-to-peer information system in Grids via self-organizing agents. Journal of Grid. Computing 6, 125–140 (2008)

    Article  Google Scholar 

  11. Forestiero, A., Mastroianni, C., Spezzano, G.: So-Grid: a self-organizing Grid featuring bio-inspired algorithms. ACM Trans. Auton. Adapt. Syst. 3(2), 1–37 (2008)

    Article  Google Scholar 

  12. Foster, I., Iamnitchi, A.: On Death, Taxes, and the Convergence of Peer-to-Peer and Grid Computing, pp. 118–128 (2003)

  13. Foster, I., Kesselman, C., Nick, J., Tuecke, S.: The physiology of the Grid: an open Grid services architecture for distributed systems integration. Technical report, Globus Project (2002)

  14. Foster, I., Kesselman, C., Tuecke,S.: The anatomy of the Grid: enabling scalable virtual organizations. Int. J. Supercomput. Appl. 15 (2001)

  15. Iamnitchi, A., Foster, I.: On fully decentralized resource discovery in Grid environments. In: GRID ’01: Proceedings of the Second International Workshop on Grid Computing, pp. 51–62. Springer, London (2001)

    Google Scholar 

  16. Jin, G., Mellor-Crummey, J.: SFCGen: a framework for efficient generation of multi-dimensional space-filling curves by recursion. ACM Trans. Math. Softw. 31(1), 120–148 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  17. Kermarrec, A., Massouli, L., Ganesh, A.: Probabilistic reliable dissemination in large-scale systems. IEEE Trans. Parallel Distrib. Syst. 14(3), 248–258 (2003)

    Article  Google Scholar 

  18. Koloniari, G., Pitoura, E.: Bloom-based filters for hierarchical data. In: 5th Workshop on Distributed Data and Structures (2003)

  19. Lawder, J.: Calculation of Mappings between One and n-dimensional Values Using the Hilbert Space-Filling Curve (2000)

  20. Li, M., Lee, W., Sivasubramaniam, A., Zhao, J.: SSW: a small-world-based overlay for peer-to-peer search. IEEE Trans. Parallel Distrib. Syst. 19(6), 735–749 (2008)

    Article  Google Scholar 

  21. Mstroianni, C., Talia, D., Verta, O.: A super-peer model for resource discovery services in large-scale Grids. Future Gener. Comput. Syst. 21(8), 1235–1248 (2005)

    Article  Google Scholar 

  22. Merz, P., Gorunova, K.: Fault-tolerant resource discovery in peer-to-peer Grids. Journal of Grid. Computing 5(3), 319–335 (2007)

    Article  Google Scholar 

  23. Moon, B., Jagadish, H.V., Faloutsos, C., Saltz, J.H.: Analysis of the clustering properties of the Hilbert space-Filling curve. IEEE Trans. Knowl. Data Eng. 13(1), 124–141 (2001)

    Article  Google Scholar 

  24. Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, New York (1995)

    MATH  Google Scholar 

  25. Newman, H.B., Legrand, I.C., Galvez, P., Voicu, R., Cirstoiu, C.: MonALISA: a distributed monitoring service architecture. In: CHEPLa Jola, California (2003)

    Google Scholar 

  26. Open Science Grid (OSG): Open Science Grid homepage. http://opensciencegrid.org (2008)

  27. Oppenheimer, D., Albrecht, J., Patterson, D., Vahdat, A.: Design and implementation tradeoffs for wide-area resource discovery. In: HPDCResearch Triangle Park, NC (2005)

    Google Scholar 

  28. Padmanabhan, A., Wang, S., Ghosh, S., Briggs, R.: A Self-Organized Grouping (SOG) method for efficient Grid resource discovery. In: GRID 2005. (2005)

  29. Ramabhadran, S., Ratnasamy, S., Hellerstein, J.M., Shenker, S.: Prefix hash tree. In: PODC (2004)

  30. Raman, R., Livny, M., Solomon, M.: Policy driven heterogeneous resource co-allocation with gangmatching. In: Proceedings of the Twelfth IEEE International Symposium on High-Performance Distributed Computing (HPDC-12)Seattle, WA (2003)

    Google Scholar 

  31. Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable content addressable network. In: Proceedings of ACM SIGCOMM (2001)

  32. RFC-Gnutella: The Gnutella Protocol Specification v0.4 (2004)

  33. Schmidt, C., Parashar, M.: Flexible information discovery in decentralized distributed systems. In: of the 12th IEEE International Symposium on High Performance Distributed Computing (HPDC’03)IEEE Computer Society, Washington, DC (2003)

    Google Scholar 

  34. Schulz, S., Blochinger, W., Hannak, H.: Capability-aware information aggregation in peer-to-peer Grids. Journal of Grid. Computing 7(2), 135–167 (2009)

    Article  Google Scholar 

  35. Spence, D., Harris, T.: XenoSearch: distributed resource discovery in the xenoServer open platform. In: Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing (HPDC’03)IEEE Computer Society, Washington, DC (2003)

    Google Scholar 

  36. Stoica, I., Morris, R., Karger, D., Kaashoek, F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup service for internet applications. In: Proceedings of the 2001 ACM SIGCOMM Conference, pp. 149–160. ACM Press, New York (2001)

    Google Scholar 

  37. Viana, A.C., de Amorim, M.D., Viniotis, Y., Fdida, S., de Rezende, J.F.: Twins: a dual addressing space representation for self-organizing networks. IEEE Trans. Parallel Distrib. Syst. 17(12), 1468–1481 (2006)

    Article  Google Scholar 

  38. Yang, B., Garcia-Molina, H.: Designing a super-peer network. In: IEEE International Conference on Data Engineering, (ICDE 2003) (2003)

  39. Zhu, C., Liu, Z., Zhang, W., Xiao, W., Xu, Z., Yang, D.: Decentralized Grid resource discovery based on resource information community. Journal of Grid. Computing 2(3), 261–277 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CyberInfrastructure and Geospatial Information Lab and National Center for Supercomputing Application (NCSA), University of Illinois at Urbana Champaign, Urbana, IL, 61801, USA

    Anand Padmanabhan & Shaowen Wang

  2. Department of Computer Science, The University of Iowa, Iowa City, IA, 52242, USA

    Sukumar Ghosh

Authors
  1. Anand Padmanabhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sukumar Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaowen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anand Padmanabhan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Padmanabhan, A., Ghosh, S. & Wang, S. A Self-Organized Grouping (SOG) Framework for Efficient Grid Resource Discovery. J Grid Computing 8, 365–389 (2010). https://doi.org/10.1007/s10723-009-9145-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-009-9145-0

Keywords

Search

Navigation