Abstract
Grid services provide an important abstract layer on top of heterogeneous components (hardware and software) that take part into a grid environment. We are developing a data grid service prototype that aims at providing transparent use of grid resources to data intensive scientific applications. Our prototype was designed having as target three main issues: (1) dynamic scheduling and allocation of query execution engine modules into grid nodes; (2)adaptability of query execution to variations on environment conditions and (3) support to special scientific operations. We propose a new node scheduling algorithm and show how it can be integrated into a simple distributed and parallel query optimization strategy. Our implementation demonstrates a speedup of 16.6 with 18 scheduled nodes and a steady throughput rate, obtained applying a dynamic adaptive strategy.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Foster, I., Kesselman, C., Nick, J., Tuecke, S.: The Physiology of the Grid: An Open Grid Services Architecture for Distributed System integration. Global Grid Forum (2002)
Foster, I., Kesselman, C.: Globus a metacomputing infrastructure toolkit. Intl. Journal Supercomputer Applications 11, 115–128 (1997)
Czajkowski, K., et al.: From open grid services infrastructure to wsresource framework: Refactoring & evolution, version 1.1. Technical report (2005), http://www.globus.org/wsrf/specs/ogsitowsrf1.0.pdf
Antonioletti, M., Atkinsons, M., et al.: The design and implementation of grid database services in ogsa-dai. Concurrency and Computation: Practice and Experience Journal 17, 357–376 (2005)
Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., Tuecke, S.: The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets (1999)
Avnur, R., Hellerstein, J.: Eddies: continuously adaptive query processing. ACM SIGMOD Record 29, 261–272 (2000)
Porto, F., Giraldi, G., de Oliveira, J.C., Schulze, B.: Codims - an adaptable middleware system for scientific visualization in grids. Concurrency and Computation: Practice and Experience Journal 16, 515–522 (2004)
Alpdemir, M.N., Mukherjee, A., Paton, N., et al.: Ogsa-dqp: A service-based distributed query processor for the grid. In: Cox, J. (ed.) Proc. of UK e-Science All Hands Meeting Nottingham (2003)
Paton, N., Atkinson, M., Dialani, V., Pearson, D., Storey, T., Watson, P.: Database access and integration services on the grid. Technical report, U.K. National eScience Center (2002), http://www.nesc.ac.uk
Smith, J., Gounaris, A., Watson, P., Paton, N.W., et al.: Distributed query processing on the grid. In: Parashar, M. (ed.) GRID 2002. LNCS, vol. 2536, pp. 279–290. Springer, Heidelberg (2002)
Vadhhiyar, S.S., Dongarra, J.J.: Self adaptivity in grid environment. Concurrency and Computation: Practice and Experience Journal 17, 235–257 (2005)
Rosenblum, L., et al. (eds.): Scientific Visualization - Advances and Challenges. Academic Press, London (1994)
Barnard, S., et al.: Large-scale distributed computational fluid dynamics on the information power grid using globus, pp. 60–67 (1999)
Molina, H., Ullman, J.D., Widow, J.D.: Database Systems: The Complete Book. Prentice Hall, Englewood Cliffs (2001)
Lo, M.L., Ravishankar, C.V.: Spatial hash-joins. In: Proc. of the ACM SIGMOD Conference on Management of Data, Montreal, Canada, pp. 247–258 (1996)
Barbosa, A., Porto, F., Melo, R.N.: Configurable data integration middleware system. Journal of the Brazilian Computer Society 8, 12–19 (2002)
Selinger, P.G., Adiba, M.E.: Access path selections in distributed data base management systems. In: Proc. 1st Intl. Conf. on Databases, British Computer Society, Aberdeen (1980)
Ozsu, M.T., Valduriez, P.: Principles of Distributed Database Systems. Prentice Hall International, Inc., New Jersey (1999)
Bouganim, L., Fabret, F., Porto, F., Valduriez, P.: Processing queries with expensive functions and large objects in distributed mediator systems. In: Proceedings of Int’l. Conf. on Data Engineering, Heidelberg, Germany, pp. 91–98 (2001)
Ayres, F., Porto, F., Melo, R.N.: An extensible query execution engine for supporting new query execution models. Technical report, EPFL, Ecole Polytechnique Fédérale de Lausanne School of Computer and Communication Sciences, Suisse (2005), http://icwww.epfl.ch/publications/documents/IC_TECH_REPORT_2005034.pdf
Porto, F.: Strategies for the Parallel execution of user programs in scientific applications. PhD thesis, Pontifícia Universidade Católica do Rio de Janeiro, RJ, Brazil (2001)
Graefe, G.: Query evaluation techniques for large databases. ACM Computing Surveys 25, 73–170 (1993)
Deering, S., Cheriton, D.: Multicast routing in datagram internetworks and extended lans. ACM Transactions on Computer Systems (8), 85–111
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Porto, F., da Silva, V.F.V., Dutra, M.L., Schulze, B. (2006). An Adaptive Distributed Query Processing Grid Service. In: Pierson, JM. (eds) Data Management in Grids. DMG 2005. Lecture Notes in Computer Science, vol 3836. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11611950_5
Download citation
DOI: https://doi.org/10.1007/11611950_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-31212-3
Online ISBN: 978-3-540-32452-2
eBook Packages: Computer ScienceComputer Science (R0)