Abstract
We address several load shedding techniques over sliding window joins. We first construct a dual window architectural model including aux-windows and join-windows, and build statistics on aux-windows. With the statistics, we develop an effective load shedding strategy producing maximum subset join outputs. In order to accelerate the load shedding process, binary indexed trees have been utilized to reduce the cost on shedding evaluation. When streams have high arrival rates, we propose an approach incorporating front-shedding and rear-shedding, and find an optimal trade-off between them. As for the scenarios of variable speed ratio, we develop a plan reallocating CPU resources and dynamically resizing the windows. In addition, we prove that load shedding is not affected during the process of reallocation. Both synthetic and real data are used in our experiments, and the results show the promise of our strategies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Babcock B, Babu S, Datar M et al. Models and issues in data stream systems. In Proc. Principles of Database Systems (PODS), June 2002.
Abadi D, Carney D et al. Aurora: A new model and architecture for data stream management. VLDB Journal, 2003, 12(2): 120–139.
Daniel J Abadi, Yanif Ahmad, M Balazinska et al. The design of the Borealis stream processing engine. In Proc. 2nd Conf. Innovative Data Systems Research, CA, USA, January 2005, pp.227–289.
The STREAM Group. STREAM: The Stanford stream data manager. IEEE Data Engineering Bulletin, March 2003, 26(1):19–26.
Chen J, DeWitt D J, Tian F et al. NiagaraCQ: A scalable continous query system for Internet databases. In Proc. ACM SIGMOD Int. Conf. Management of Data, Dallas, USA, 2000, pp.379–390.
Hellerstein J M, Franklin M J, Chandrasekaran S et al. Adaptive query processing: Technology in evolution. IEEE Data Engineering Bulletin, 2000, 23(2): 7–18.
Kang J, Naughton J F, Viglas S D. Evaluating window joins over unbounded streams. In Proc. 2003 Int. Conf. Data Engineering, Mar. 2003, pp.341–352.
Ayad A M, Naughton J F. Static optimization of conjunctive queries with sliding windows over infinite streams. In Proc. ACM SIGMOD Conf., Paris, France, June 2004, pp.419–430.
Das A, Gehrke J, Riedewald M. Approximate join processing over data streams. In Proc. 2003 ACM SIGMOD Conf., San Diego, June 2003, pp.40–51.
Xie J, Yang J, Chen Y. On joining and caching stochastic streams. In Proc. 2005 ACM SIGMOD Conf., Baltimore, Maryland, USA, June 2005, pp.359–370.
Srivastava U, Widom J. Memory-limited execution of windowed stream joins. In Proc. 30th Int. Conf. Very Large Data Bases, 2004.
Fenwich P M. A new data structure for cumulative frequency tables. Software – Practice and Experience, Mar. 1994, 24(30): 327–336.
Han D, Xiao C, Zhou R et al. Load shedding for Window Joins over data streams. In Proc. Conf. Web-Age Information Management, Hong Kong, June 2006, pp.397–409.
Baldocchi D, Wilson K et al. Half-hourly measurements of CO2, water vapor, and energy exchange using the Eddy covariance technique from walker branch watershed, Tennessee, 1995–1998, http://cdiac.esd.ornl.gov/ftp/ameriflux/data/us-sites/walker-branch/.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work is supported by the National Natural Science Foundation of China under Grant Nos. 60473074, 60573089 and National Grand Fundamental Research 973 Program of China under Grant No. 2006CB303103.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Han, DH., Wang, GR., Xiao, C. et al. Load Shedding for Window Joins over Streams. J Comput Sci Technol 22, 182–189 (2007). https://doi.org/10.1007/s11390-007-9024-8
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11390-007-9024-8