Yongluan Zhou
ABSTRACT
Many data stream applications, such as network intrusion detection, on-line financial tickers and environmental monitoring, typically exhibit certain "real-time" traits. In such applications, people are interested in strategies that ensure on-time delivery of query results. In this paper, we point out that traditional operator-based query scheduling strategies are insufficient to handle this class of problem. Therefore we choose to approach the issue from a new angle by modeling multi-query scheduling as a job-scheduling problem, a classical problem in real-time computing. By taking advantage of the wisdom in the real-time computing community, we propose several new scheduling strategies and algorithms to enhance the overall data stream query scheduling performance. Through extensive experiments over both real and synthetic data, we identify the important factors for scheduling performance and verify the effectiveness of our approaches.
- Internet traffic archive. http://www.sigcomm.org/ITA.Google Scholar
- R. Avnur and J. M. Hellerstein. Eddies: Continuously adaptive query processing. In SIGMOD Conference, pages 261--272, 2000. Google ScholarDigital Library
- B. Babcock, S. Babu, M. Datar, and R. Motwani. Chain: Operator scheduling for memory minimization in data stream systems. In SIGMOD Conference, pages 253--264, 2003. Google ScholarDigital Library
- S. K. Baruah, G. Koren, D. Mao, B. Mishra, A. Raghunathan, L. E. Rosier, D. Shasha, and F. Wang. On the competitiveness of on-line real-time task scheduling. Real-Time Systems, 4(2):125--144, 1992. Google ScholarDigital Library
- D. Carney, U. Çetintemel, A. Rasin, S. B. Zdonik, M. Cherniack, and M. Stonebraker. Operator scheduling in a data stream manager. In VLDB, pages 838--849, 2003. Google ScholarDigital Library
- B. Gedik, K.-L. Wu, P. S. Yu, and L. Liu. Adaptive load shedding for windowed stream joins. In CIKM, 2005. Google ScholarDigital Library
- G. V. Gens and E. V. Levner. Fast approximation algorithm for job sequencing with deadlines. Discrete Applied Mathematics, 3:313--318, 1981.Google ScholarCross Ref
- R. Givan, E. K. P. Chong, and H. S. Chang. Scheduling multiclass packet streams to minimize weighted loss. Queueing Syst., 41(3):241--270, 2002. Google ScholarDigital Library
- Q. Jiang and S. Chakravarthy. Scheduling strategies for processing continuous queries over streams. In BNCOD, pages 16--30, 2004.Google ScholarCross Ref
- R. M. Karp. Reducibility among combinatorial problems. In Complexity of Computer Computations, pages 85--103. Plenum, New York, 1972.Google ScholarCross Ref
- E. L. Lawler. Sequencing to minimize the weighted number of tardy jobs. RAIRO Operations Research, 10:27--33, 1976.Google Scholar
- E. L. Lawler and J. Moore. A functional equation and its application to resource allocation and sequencing problems. Management Science, 16:77--84, 1969.Google ScholarDigital Library
- W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson. On the self-similar nature of ethernet traffic (extended version). IEEE/ACM Trans. Netw., 2(1):1--15, 1994. Google ScholarDigital Library
- C. D. Locke. Best-Effort Decision Making for Real-Time Scheduling. PhD thesis, Carnegie Mellon University, 1986. Google ScholarDigital Library
- J. Moore. An n job one machine sequencing algorithm for minimizing the number of late jobs. Management Science, 15(1):102--109, 1968.Google ScholarDigital Library
- V. Paxson and S. Floyd. Wide area traffic: the failure of poisson modeling. IEEE/ACM Trans. Netw., 3(3):226--244, 1995. Google ScholarDigital Library
- S. Sahni. Algorithms for scheduling independent tasks. J. ACM, 23(1):116--127, 1976. Google ScholarDigital Library
- M. A. Sharaf, P. K. Chrysanthis, A. Labrinidis, and K. Pruhs. Efficient scheduling of heterogeneous continuous queries. In VLDB, pages 511--522, 2006. Google ScholarDigital Library
- U. Srivastava and J. Widom. Flexible time management in data stream systems. In PODS, pages 263--274, 2004. Google ScholarDigital Library
- N. Tatbul, U. Çetintemel, S. B. Zdonik, M. Cherniack, and M. Stonebraker. Load shedding in a data stream manager. In VLDB, pages 309--320, 2003. Google ScholarDigital Library
- P. A. Tucker, D. Maier, T. Sheard, and L. Fegaras. Exploiting punctuation semantics in continuous data streams. IEEE Trans. Knowl. Data Eng., 15(3):555--568, 2003. Google ScholarDigital Library
- T. Urhan and M. J. Franklin. Dynamic pipeline scheduling for improving interactive query performance. In VLDB, pages 501--510, 2001. Google ScholarDigital Library
- M. Wang, N. H. Chan, S. Papadimitriou, C. Faloutsos, and T. M. Madhyastha. Data mining meets performance evaluation: Fast algorithms for modeling bursty traffic. In ICDE, pages 507--516, 2002. Google ScholarDigital Library
- J. Wu, K.-L. Tan, and Y. Zhou. Qos-oriented multi-query scheduling over data streams. In DASFAA, pages 215--229, 2009. Google ScholarDigital Library
- Y. Zhou, J. Wu, and A. K. Leghari. Multi-query scheduling for time-critical data stream applications. Unpublished manuscript available from the authors, 2013.Google Scholar
Index Terms
- Multi-query scheduling for time-critical data stream applications
Recommendations
Condensative stream query language for data streams
ADC '07: Proceedings of the eighteenth conference on Australasian database - Volume 63In contrast to traditional database queries, a query on stream data is continuous in that it is periodically evaluated over fractions (sliding windows) of the data stream. This introduces challenges beyond those encountered when processing traditional ...
Real-Time Scheduling for Data Stream Management Systems
ECRTS '05: Proceedings of the 17th Euromicro Conference on Real-Time SystemsQuality-aware management of data streams is gaining moreand more importance with the amount of data produced by streams growing continuously. The resources required for data stream processing depend on different factors and are limited by the ...
Comments