Abstract
In distributed scientific query processing systems, leveraging distributed cached data is becoming more important. In such systems, a front-end query scheduler distributes queries among many application servers rather than processing queries in a few high-performance workstations. Although many query scheduling policies exist such as round-robin and load-monitoring, they are not sophisticated enough to exploit cached results as well as balance the workload. Efforts were made to improve the query processing performance using statistical methods such as exponential moving average. However, existing methods have limitations for certain query patterns: queries with hotspots, or dynamic query distributions. In this paper, we propose novel query scheduling policies that take into account both the contents of distributed caching infrastructure and the load balance among the servers. Our experiments show that the proposed query scheduling policies outperform existing policies by producing better query plans in terms of load balance and cache-hit ratio.
Similar content being viewed by others
References
Andrade, H., Kurc, T., Sussman, A., Saltz, J.: Multiple query optimization for data analysis applications on clusters of SMPs. In: Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid). IEEE Comput. Soc., Los Alamitos (2002)
Aron, M., Sanders, D., Druschel, P., Zwaenepoel, W.: Scalable content-aware request distribution in cluster-based network servers. In: Proceedings of Usenix Annual Technical Conference (2000)
Arya, S., Mount, D.M., Netanyahu, N.S., Silverman, R., Wu, A.Y.: An optimal algorithm for approximate nearest neighbor searching fixed dimensions. J. ACM 45(6), 891–923 (1998)
de Berg, M., Cheong, O., van Kreveld, M., Overmars, M.: Computational Geometry, Algorithms and Applications. Springer, Berlin (1998)
Chen, F.C.F., Dunham, M.H.: Common subexpression processing in multiple-query processing. Transactions on Knowledge and Data Engineering 10(5), 493–499 (199)
Chou, Y.l.: Statistical Analysis. Holt International (1975)
Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)
Menasce, D.A., Almeida, V.A.F.: Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning. Prentice Hall, New York (2000)
Nam, B., Shin, M., Andrade, H., Sussman, A.: Multiple query scheduling for distributed semantic caches. J. Parallel Distrib. Comput. 70(5), 598–611 (2010)
Pai, V., Aron, M., Banga, G., Svendsen, M., Druschel, P., Zwaenepoel, W., Nahum, E.: Locality-aware request distribution in cluster-based network servers. In: Proceedings of ACM ASPLOS (1998)
Ren, Q., Dunham, M.H., Kumar, V.: Semantic caching and query processing. IEEE Trans. Knowl. Data Eng. 15(1), 192–210 (2003)
Roy, P., Sehadri, S., Sudarshan, S., Bhobe, S.: Efficient and extensible algorithms for multi query optimization. In: Proceedings of 2000 ACM SIGMOD International Conference on Management of Data (SIGMOD), pp. 249–260 (2000)
Sellis, T.K., Ghosh, S.: On the multiple-query optimization problem. IEEE Trans. Knowl. Data Eng. 2(2), 262–266 (1990)
Xiong, X., Mokbel, M.F., Aref, W.G., Hambrusch, S.E., Prabhakar, S.: Scalable spatio-temporal continuous query processing for location-aware services. In: Proceedings of 16th International Conference on Scientific and Statistical Database Management (SSDBM) (2004)
Zhang, K., Andrade, H., Raschid, L., Sussman, A.: Query planning for the Grid: adapting to dynamic resource availability. In: Proceedings of the 5th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid), Cardiff, UK (2005)
Zhao, Y., Desshpande, P.M., Naughton, J.F., Shukla, A.: Simultaneous optimization and evaluation of multiple dimensional queries. In: Proceedings of 1998 ACM SIGMOD International Conference on Management of Data (SIGMOD), pp. 271–282 (1998)
Acknowledgements
This research was supported by PLSI resources, 1.100027.01 Research Fund of the UNIST (Ulsan National Institute of Science and Technology), and 2.110147.01 National Research Foundation of Korea. This work was also supported by 2011 Research Fund of Myongji University.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Judy Qiu and Dennis Gannon.
Rights and permissions
About this article
Cite this article
Nam, B., Hwang, D., Kim, J. et al. High-throughput query scheduling with spatial clustering based on distributed exponential moving average. Distrib Parallel Databases 30, 401–414 (2012). https://doi.org/10.1007/s10619-012-7098-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10619-012-7098-y