Skip to main content
SpringerLink
Log in

Decentralized multi-attribute range search for resource discovery and load balancing

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The recent trend toward heterogeneity among, and even within, computers requires new expressiveness in the way resource descriptions are created, and new sophistication in systems that attempt to match jobs with resources. In this paper, we propose a novel resource discovery and load balancing method to support multi-attribute, range-based job constraints in a peer-to-peer grid system. The common approach of using simple attribute indexes does not suffice, as range-based constraints may be satisfied by more than a single value. We show via simulation that our approach matches these new heterogeneous jobs to heterogeneous resources both effectively (good matches are found, load is balanced) and efficiently (the new functionality imposes little overhead). Our system relies on resource descriptions being aggregated and periodically disseminated. Decentralized algorithms are often sensitive to the “freshness” of this information. However, we show that our approach is not sensitive to stale data, allowing the information movement to happen rarely enough that it imposes only insignificant overhead.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Bharambe AR, Agrawal M, Seshan S (2004) Mercury: supporting scalable multi-attribute range queries. In: Proceedings of the ACM SIGCOMM Conference

  2. Bloom BH (1970) Space/time trade-offs in hash coding with allowable errors. Commun ACM 13:422–426

    Article  MATH  Google Scholar 

  3. Ganesan P, Yang B, Garcia-Molina H (2004) One torus rule them all: multi-dimensional queries in P2P systems. In: Proceedings of the 7th international workshop on web and databases

  4. Jagadish H, Ooi BC, Vu QH, Zhang R, Zhou A (2006) Vbi-tree: a peer-to-peer framework for supporting multi-dimensional indexing schemes. In: Proceedings of the 22nd international conference on data engineering, 2006. ICDE ’06, p 34. doi:10.1109/ICDE.2006.169

  5. Kim JS, Keleher P, Marsh M, Bhattacharjee B, Sussman A (2007) Using content-addressable networks for load balancing in desktop grids. In: IEEE international symposium on high performance distributed computing (HPDC)

  6. Kim JS, Nam B, Keleher P, Marsh M, Bhattacharjee B, Sussman A (2006) Resource discovery techniques in distributed desktop grid environments. In: Proceedings of the 7th IEEE/ACM international conference on grid computing-GRID 2006

  7. Kim JS, Nam B, Marsh M, Keleher P, Bhattacharjee B, Sussman A (2008) Integrating categorical resource types into a P2P desktop grid system. In: Proceedings of the 9th IEEE/ACM international conference on grid computing (GRID 2008)

  8. Lawder J (2000) Calculation of mappings between one and n-dimensional values using the Hilbert space-filling curve. Tech. Rep. BBKCS-00-01, Birkbeck College

  9. Lee J, Keleher P, Sussman A (2010) Decentralized resource management for multi-core desktop grids. In: Proceedings of the 24th IEEE international parallel & distributed processing symposium

  10. Lee J, Keleher P, Sussman A (2011) Supporting computing element heterogeneity in p2p grids. In: 2011 IEEE international conference on cluster computing (CLUSTER), pp 150–158. doi:10.1109/CLUSTER.2011.25

  11. Li X, Kim YJ, Govindan R, Hong W (2003) Multi-dimensional range queries in sensor networks. In: Proceedings of the 1st international conference on embedded networked sensor systems, SenSys ’03. ACM, New York, pp 63–75. doi:10.1145/958491.958500

  12. Maheswaran M, Ali S, Siegel HJ, Hensgen D, Freud RF (1999) Dynamic mapping of a class of independent tasks onto heterogeneous computing systems. J Parallel Distrib Comput 59(2)

  13. Nam B, Sussman A (2006) DiST: fully decentralized indexing for querying distributed multidimensional datasets. In: Proceedings of IPDPS 2006

  14. Oppenheimer D, Albrecht J, Patterson D, Vahdat A (2005) Design and implementation tradeoffs for wide-area resource discovery. In: Proceedings 14th IEEE international symposium on high performance distributed computing, HPDC-14, pp 113–124. doi:10.1109/HPDC.2005.1520946

  15. Ratnasamy S, Francis P, Handley M, Karp R, Shenker S (2001) A scalable content addressable network. In: Proceedings of the ACM SIGCOMM conference

  16. Rowstran A, Druschel P (2001) Pastry: scalable, distributed object location and routing for large-scale peer-to-peer systems. In: Proceedings of the 18th IFIP/ACM international conference on distributed systems platforms (Middleware 2001)

  17. Sahin OD, Gulbeden A, Emekci F, Agrawal D, Abbadi AE (2005) Prism: indexing multi-dimensional data in p2p networks using reference vectors. In. In MULTIMEDIA ’05: Proceedings of the 13th annual ACM international conference on multimedia. ACM Press, New York, pp 946–955

  18. Stoica I, Morris R, Karger D, Kaashoek MF, Balakrishnan H (2001) Chord: a scalable peer-to-peer lookup service for internet applications. In: Proceedings of the ACM SIGCOMM Conference

  19. Tang Y, Xu J, Zhou S, Lee W (2009) m-light: Indexing multi-dimensional data over dhts. In: 29th IEEE international conference on distributed computing systems, 2009. ICDCS ’09, pp 191–198. doi:10.1109/ICDCS.2009.30

  20. Tanin E, Harwood A, Samet H (2007) Using a distributed quadtree index in peer-to-peer networks. VLDB J 16:165–178. doi:10.1007/s00778-005-0001-y

    Google Scholar 

  21. Vo HT, Chen C, Ooi BC (2010) Towards elastic transactional cloud storage with range query support. Proc VLDB Endow 3(1–2):506–514

    Google Scholar 

  22. Wang J, Wu S, Gao H, Li J, Ooi BC (2010) Indexing multi-dimensional data in a cloud system. In: Proceedings of the 2010 international conference on Management of data, SIGMOD ’10. ACM, New York, pp 591–602. doi:10.1145/1807167.1807232

  23. Zhao BY, Huang L, Stribling J, Rhea SC, Joseph AD, Kubiatowicz JD (2004) Tapestry: a resilient global-scale overlay for service deployment. IEEE J Sel Areas Commun 22(1)

Download references

Author information

Authors and Affiliations

  1. UMIACS and Department of Computer Science, University of Maryland, College Park, MD, 20742, USA

    Jaehwan Lee, Pete Keleher & Alan Sussman

Authors
  1. Jaehwan Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pete Keleher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alan Sussman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jaehwan Lee.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lee, J., Keleher, P. & Sussman, A. Decentralized multi-attribute range search for resource discovery and load balancing. J Supercomput 68, 890–913 (2014). https://doi.org/10.1007/s11227-013-1071-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-013-1071-z

Keywords

Search

Navigation