Abstract
Data streaming applications are becoming more and more common due to the rapid development in emerging areas such as sensor networks, multimedia streaming, and on-line data mining, etc. These applications are often running in a decentralized, distributed environment. The requirements for processing large volumes of streaming data at real time have posed many great design challenges. One of the critical issues is to optimize the ongoing resource consumption of multiple, distributed, cooperating processing units. In this paper, we consider a generic model for the general stream data processing systems. We address the resource allocation problem for a collection of processing units so as to maximize the weighted sum of the throughput of different streams. Each processing unit may require multiple input data streams simultaneously and produce one or many valuable output streams. We develop decentralized control mechanisms that maximize the overall system throughput in such data stream processing networks. Performance analysis on the optimality and complexity of these mechanisms are also provided.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baccelli, F., & Liu, Z. (1989). On the stability condition of a precedence-based queueing discipline. Advances in Applied Probability, 21, 883–887.
Baccelli, F., & Liu, Z. (1990). On the execution of parallel programs on multiprocessor systems—a queuing theory approach. Journal of the Association for Computing Machinery, 37(2), 373–417.
Baldwin, C., Clark, K. B., Magretta, J., Dyer, J. H., Fisher, M., & Fites, D. V. (2000). Harvard business review on managing the value chain. Boston: Harvard Business School Press.
Brandstädt, A., Bang Le, V., & Spinrad, J. P. (1999). Graph classes: a survey. Philadelphia: SIAM.
Mas-Collel, A., Whinston, M., & Green, J. (1995). Microeconomic theory. London: Oxford University Press.
Simchi-Levi, D., Kaminsky, P., & Simchi-Levi, E. (2002). Designing and managing the supply chain (2nd ed.). New York: McGraw-Hill.
Viglas, S., & Naughton, J. (2002). Rate-based query optimization for streaming information sources. In ACM SIGMOD.
Xi, B., Liu, Z., Raghavachari, M., Xia, C., & Zhang, L. (2004). A smart hill-climbing algorithm for application server configuration. In Proc. of the 13th international conference on world wide web (pp. 287–296).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, Z., Tang, A., Xia, C.H. et al. A decentralized control mechanism for stream processing networks. Ann Oper Res 170, 161–182 (2009). https://doi.org/10.1007/s10479-008-0434-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-008-0434-y