Gabriele Russo Russo
Valeria Cardellini
Francesco Lo Presti
ABSTRACT
Data Stream Processing (DSP) has emerged as a key enabler to develop pervasive services that require to process data in a near real-time fashion. DSP applications keep up with the high volume of produced data by scaling their execution on multiple computing nodes, so as to process the incoming data flow in parallel. Workloads variability requires to elastically adapt the application parallelism at run-time in order to avoid over-provisioning. Elasticity policies for DSP have been widely investigated, but mostly under the simplifying assumption of homogeneous infrastructures. The resulting solutions do not capture the richness and inherent complexity of modern infrastructures, where heterogeneous computing resources are available on-demand. In this paper, we formulate the problem of controlling elasticity on heterogeneous resources as a Markov Decision Process (MDP). The resulting MDP is not easily solved by traditional techniques due to state space explosion, and thus we show how linear Function Approximation and Tile Coding can be used to efficiently compute elasticity policies at run-time. In order to deal with parameters uncertainty, we integrate the proposed approach with Reinforcement Learning algorithms. Our numerical evaluation shows the efficacy of the presented solutions compared to standard methods in terms of accuracy and convergence speed.
- Y. Al-Dhuraibi, F. Paraiso, N. Djarallah, and P. Merle. 2018. Elasticity in Cloud Computing: State of the Art and Research Challenges. IEEE Trans. Serv. Comput. 11 (2018), 430--447.Google Scholar
Cross Ref
- V. Cardellini, F. Lo Presti, M. Nardelli, and G. Russo Russo. 2018. Decentralized Self-Adaptation for Elastic Data Stream Processing. Future Gener. Comput. Syst. 87 (2018), 171--185.Google ScholarDigital Library
- V. Cardellini, F. Lo Presti, M. Nardelli, and G. Russo Russo. 2018. Optimal Operator Deployment and Replication for Elastic Distributed Data Stream Processing. Concurr. Comput.: Pract. Exper. 30, 9 (2018), e4334.Google ScholarCross Ref
- M.D. de Assunção, A. da Silva Veith, and R. Buyya. 2018. Distributed data stream processing and edge computing: A survey on resource elasticity and future directions. J. Netw. Comput. Appl. 103 (2018), 1--17. Google ScholarDigital Library
- T. De Matteis and G. Mencagli. 2017. Proactive Elasticity and Energy Awareness in Data Stream Processing. J. Syst. Softw. 127 (2017), 302--319. Google ScholarDigital Library
- R.C. Fernandez, M. Migliavacca, E. Kalyvianaki, and P. Pietzuch. 2013. Integrating Scale Out and Fault Tolerance in Stream Processing Using Operator State Management. In Proc. ACM SIGMOD '13. 725--736. Google ScholarDigital Library
- B. Gedik, S. Schneider, M Hirzel, and K. Wu. 2014. Elastic Scaling for Data Stream Processing. IEEE Trans. Parallel Distrib. Syst. 25, 6 (2014), 1447--1463. Google ScholarDigital Library
- A. Geramifard, T.J. Walsh, S. Tellex, G. Chowdhary, N. Roy, J.P. How, et al. 2013. A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning. Found. Trends in Mach. Learn. 6, 4 (2013), 375--451. Google ScholarDigital Library
- P. Graubner, C. Thelen, M. Körber, A. Sterz, G. Salvaneschi, et al. 2018. Multimodal Complex Event Processing on Mobile Devices. In Proc. ACM DEBS '18. 112--123.Google ScholarDigital Library
- V. Gulisano, R. Jiménez-Peris, M. Patiño Martinez, C. Soriente, and P. Valduriez. 2012. StreamCloud: An Elastic and Scalable Data Streaming System. IEEE Trans. Parallel Distrib. Syst. 23, 12 (2012), 2351--2365. Google ScholarDigital Library
- J. He, Y. Chen, T. Z. J. Fu, X. Long, M. Winslett, L. You, and Z. Zhang. 2018. HaaS: Cloud-Based Real-Time Data Analytics with Heterogeneity-Aware Scheduling. In Proc. IEEE ICDCS '18. 1017--1028.Google Scholar
- T. Heinze, L. Aniello, L. Querzoni, and J. Zbigniew. 2014. Cloud-based Data Stream Processing. In Proc. ACM DEBS '14. 238--245. Google ScholarDigital Library
- T. Heinze, V. Pappalardo, Z. Jerzak, and C. Fetzer. 2014. Auto-scaling Techniques for Elastic Data Stream Processing. In Proc. IEEE ICDEW '14. 296--302.Google Scholar
- M. Hirzel, R. Soulé, S. Schneider, B. Gedik, and R. Grimm. 2014. A Catalog of Stream Processing Optimizations. ACM Comput. Surv. 46, 4 (2014), 46:1--46:34. Google ScholarDigital Library
- Z. Jerzak and H. Ziekow. 2015. The DEBS 2015 Grand Challenge. In Proc. ACM DEBS '15. ACM, 266--268.Google Scholar
- A. Koliousis, M. Weidlich, R. Castro Fernandez, A.L. Wolf, P. Costa, and P. Pietzuch. 2016. SABER: Window-Based Hybrid Stream Processing for Heterogeneous Architectures. In Proc. ACM SIGMOD '16. 555--569. Google ScholarDigital Library
- R. M. Kretchmar and C. W. Anderson. 1997. Comparison of CMACs and Radial Basis Functions for Local Function Approximators in Reinforcement Learning. In Proc. ICNN '97, Vol. 2. 834--837.Google Scholar
- G. T. Lakshmanan, Y. Li, and R. Strom. 2008. Placement Strategies for Internet-scale Data Stream Systems. IEEE Internet Comput. 12, 6 (2008), 50--60. Google ScholarDigital Library
- X. Liu, A.V. Dastjerdi, R.N. Calheiros, C. Qu, and R. Buyya. 2018. A Stepwise Auto-Profiling Method for Performance Optimization of Streaming Applications. ACM Trans. Auton. Adapt. Syst. 12, 4 (2018), 24:1--24:33. Google ScholarDigital Library
- B. Lohrmann, P. Janacik, and O. Kao. 2015. Elastic Stream Processing with Latency Guarantees. In Proc. IEEE ICDCS '15. 399--410.Google Scholar
- F. Lombardi, L. Aniello, S. Bonomi, and L. Querzoni. 2018. Elastic Symbiotic Scaling of Operators and Resources in Stream Processing Systems. IEEE Trans. Parallel Distrib. Syst. 29, 3 (2018), 572--585.Google ScholarCross Ref
- G. Mencagli. 2016. A Game-Theoretic Approach for Elastic Distributed Data Stream Processing. ACM Trans. Auton. Adapt. Syst. 11, 2 (2016), 13:1--13:34.Google ScholarDigital Library
- M.A.U. Nasir, G. De Francisci Morales, D. García-Soriano, N. Kourtellis, and M. Serafini. 2015. The Power of Both Choices: Practical Load Balancing for Distributed Stream Processing Engines. In Proc. IEEE ICDE '15. 137--148.Google Scholar
- M.L. Puterman. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons. Google Scholar
- G. Russo Russo, M. Nardelli, V. Cardellini, and F. Lo Presti. 2018. Multi-Level Elasticity for Wide-Area Data Streaming Systems: A Reinforcement Learning Approach. Algorithms 11, 9 (2018), 134.Google Scholar
- F. Starks, V. Goebel, S. Kristiansen, and T. Plagemann. 2018. Mobile Distributed Complex Event Processing---Ubi Sumus? Quo Vadimus? In Mobile Big Data: A Roadmap from Models to Technologies. Springer, 147--180.Google Scholar
- R.S. Sutton. 1995. Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding. In Proc. NIPS '95. MIT Press, 1038--1044. Google ScholarDigital Library
- R.S. Sutton and A.G. Barto. 1998. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, USA. Google ScholarDigital Library
- C. Watkins and P. Dayan. 1992. Q-learning. Machine Learning 8, 3-4 (1992), 279--292. Google ScholarDigital Library
- K.P. Yoon and C.-L. Hwang. 1995. Multiple Attribute Decision Making: an Introduction. Sage Pubs.Google Scholar
Index Terms
- Reinforcement Learning Based Policies for Elastic Stream Processing on Heterogeneous Resources
Recommendations
Reinforcement learning with Gaussian processes for condition-based maintenance
Highlights- Reinforcement learning for condition-based maintenance with continuous-state MDP.
- Gaussian process regression for function approximation in reinforcement learning.
- Develop a new Gaussian process for reinforcement learning (GPRL) ...
AbstractCondition-based maintenance strategies are effective in enhancing reliability and safety for complex engineering systems that exhibit degradation phenomena with uncertainty. Such sequential decision-making problems are often modeled as Markov ...
Risk-sensitive reinforcement learning: a martingale approach to reward uncertainty
ICAIF '20: Proceedings of the First ACM International Conference on AI in FinanceWe introduce a novel framework to account for sensitivity to rewards uncertainty in sequential decision-making problems. While risk-sensitive formulations for Markov decision processes studied so far focus on the distribution of the cumulative reward as ...
Reinforcement learning algorithms: A brief survey
Highlights- RL can be used to solve problems involving sequential decision-making.
- RL is based on trial-and-error learning through rewards and punishments.
- The ultimate goal of an RL agent is to maximize cumulative reward.
- RL agent tries ...
AbstractReinforcement Learning (RL) is a machine learning (ML) technique to learn sequential decision-making in complex problems. RL is inspired by trial-and-error based human/animal learning. It can learn an optimal policy autonomously with knowledge ...
Comments