Abstract
In data stream management systems (DSMSs), Quality of Service (or QoS) requirements, as specified by users, are extremely important. To satisfy QoS requirements throughout the life of a data stream, result characteristics need to be monitored at runtime and adjustments made continuously. It has been shown that in a DSMS, switching scheduling strategies at runtime can change tuple latency requirements. DSMSs also experience significant fluctuations in input rates (termed bursty inputs). In order to meet the QoS requirements in the presence of bursty inputs, a load shedding strategy is critical. This also entails monitoring of QoS measures at run-time to meet expected QoS requirements.
This paper addresses load shedding issues for MavStream, a DSMS being developed at UT Arlington. To cope with situations where the arrival rates of input streams exceed the processing capacity of the system, we have incorporated load shedders into the query processing model. The runtime optimizer continually monitors the output and decides when to turn on the shedders and how much to shed. Choice of shedders is done to minimize the error in the output. Shedders have been incorporated as part of the buffers to minimize the overhead for load shedding. Finally, load shedders are activated and deactivated dynamically by the runtime optimizer. Both random and semantic load shedding techniques are supported to match application semantics.
The work done in this paper is currently supported by NSF IIS - 0534611, NSF IIS - 0326505 and NSF EIA - 0216500.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Balakrishnan, H., Balazinska, M., Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Galvez, E.F., Salz, J., Stonebraker, M., Tatbul, N., Tibbetts, R., Zdonik, S.B.: Retrospective on aurora. VLDB J. 13(4), 370–383 (2004)
Babcock, B., Datar, M., Motwani, R.: Load shedding for aggregation queries over data streams. In: ICDE, pp. 350–361 (2004)
Chandrasekaran, S., Cooper, O., Deshpande, A., Franklin, M.J., Hellerstein, J.M., Hong, W., Krishnamurthy, S., Madden, S., Reiss, F., Shah, M.A.: Telegraphcq: Continuous dataflow processing. In: SIGMOD Conference, p. 668 (2003)
Gilani, A., Sonune, S., Kendai, B., Chakravarthy, S.: The Anatomy of a Stream Processing System. In: Bell, D.A., Hong, J. (eds.) BNCOD 2006. LNCS, vol. 4042, pp. 232–239. Springer, Heidelberg (2006)
Jiang, Q., Chakravarthy, S.: Scheduling strategies for processing continuous queries over streams. In: Williams, H., MacKinnon, L.M. (eds.) BNCOD 2004. LNCS, vol. 3112, pp. 16–30. Springer, Heidelberg (2004)
Arasu, A., Widom, J.: Resource sharing in continuous sliding-window aggregates. In: VLDB, pp. 336–347 (2004)
Tatbul, N., Çetintemel, U., Zdonik, S.B., Cherniack, M., Stonebraker, M.: Load shedding in a data stream manager. In: VLDB, pp. 309–320 (2003)
Jiang, Q., Chakravarthy, S.: Load shedding in a data stream management system. TR CSE-2003, UT Arlington (November 2003)
Srivastava, U., Widom, J.: Memory-limited execution of windowed stream joins. In: VLDB, pp. 324–335 (2004)
Shah, M.A., Chandrasekaran, S.: Fault-tolerant, Load-balancing Queries in Telegraph. In: SIGMOD Conference, p. 611 (2001)
Madden, S., Franklin, M.J., Hellerstein, J.M., Hong, W.: TinyDB: an acquisitional query processing system for sensor networks. ACM TODS 30(1), 122–173 (2005)
Chakravarthy, S., Pajjuri, V.: Scheduling strategies and their evaluation in a data stream management system. In: Bell, D.A., Hong, J. (eds.) BNCOD 2006. LNCS, vol. 4042, pp. 220–231. Springer, Heidelberg (2006)
Qingchun, J.: A framework for supporting quality of service requirements in a data stream management system. Ph.D. dissertation, University of Texas at Arlington, Arlington (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kendai, B., Chakravarthy, S. (2008). Load Shedding in MavStream: Analysis, Implementation, and Evaluation. In: Gray, A., Jeffery, K., Shao, J. (eds) Sharing Data, Information and Knowledge. BNCOD 2008. Lecture Notes in Computer Science, vol 5071. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70504-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-70504-8_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70503-1
Online ISBN: 978-3-540-70504-8
eBook Packages: Computer ScienceComputer Science (R0)