skip to main content
10.1145/3194133.3194152acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Adapting a system with noisy outputs with statistical guarantees

Published: 28 May 2018 Publication History

Abstract

Many complex systems are intrinsically stochastic in their behavior which complicates their control and optimization. Current self-adaptation and self-optimization approaches are not tailored to systems that have (i) complex internal behavior that is unrealistic to model explicitly, (ii) noisy outputs, (iii) high cost of bad adaptation decisions, i.e. systems that are both hard and risky to adapt at runtime. In response, we propose to model the system to be adapted as black box and apply state-of-the-art optimization techniques combined with statistical guarantees. Our main contribution is a framework that combines runtime optimization with guarantees obtained from statistical testing and with a method for handling cost of bad adaptation decisions. We evaluate the feasibility of our approach by applying it on an existing traffic navigation self-adaptation exemplar.

References

[1]
Cheng, S.-W. et al. 2012. Stitch: A language for architecture-based self-adaptation. Journal of Systems and Software. 85, 12 (2012), 1--38.
[2]
Elkhodary, A. et al. 2010. FUSION: A Framework for Engineering Self-tuning Self-adaptive Software Systems. Proc. of FSE '10 (2010), 7--16.
[3]
Esfahani, N. et al. 2013. A Learning-Based Framework for Engineering Feature-Oriented Self-Adaptive Software Systems. IEEE Transactions on Software Engineering. 39, 11 (Nov. 2013), 1467--1493.
[4]
Filieri, A. et al. 2014. Automated Design of Self-adaptive Software with Control-theoretical Formal Guarantees. Proceedings of the 36th International Conference on Software Engineering (New York, NY, USA, 2014), 299--310.
[5]
Garlan, D. et al. 2004. Rainbow: Architecture-Based Self-Adaptation with Reusable Infrastructure. Computer. 37, 10 (2004), 46--54.
[6]
Ghezzi, C. et al. 2013. Managing Non-functional Uncertainty via Model-driven Adaptivity. Proc. of ICSE'13 (2013), 33--42.
[7]
Golovin, D. et al. 2017. Google Vizier: A Service for Black-Box Optimization. (2017), 1487--1495.
[8]
Iftikhar, M.U. and Weyns, D. 2014. ActivFORMS: Active Formal Models for Self-adaptation. Proc. SEAMS '14 (2014), 125--134.
[9]
Jamshidi, P. et al. 2017. Transfer Learning for Improving Model Predictions in Highly Configurable Software. arXiv:1704.00234 {cs} (Apr. 2017).
[10]
Jamshidi, P. and Casale, G. 2016. An uncertainty-aware approach to optimal configuration of stream processing systems. Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS), 2016 IEEE 24th International Symposium on (2016), 39--48.
[11]
Kohavi, R. et al. 2009. Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery. 18, 1 (Feb. 2009), 140--181.
[12]
Kreiss, J.-P. and Paparoditis, E. 2011. Bootstrap methods for dependent data: A review. Journal of the Korean Statistical Society. 40, 4 (Dec. 2011), 357--378.
[13]
Lewis, R.M. et al. 2000. Direct search methods: then and now. Journal of Computational and Applied Mathematics. 124, 1 (Dec. 2000), 191--207.
[14]
Mattos, D.I. et al. 2017. More for Less: Automated Experimentation in Software-Intensive Systems. Product-Focused Software Process Improvement (Nov. 2017), 146--161.
[15]
Mattos, D.I. et al. 2017. Your System Gets Better Every Day You Use It: Towards Automated Continuous Experimentation. 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA) (Aug. 2017), 256--265.
[16]
Nocedal, J. and Wright, S.J. 2006. Numerical Optimization. Springer.
[17]
Pan, S.J. and Yang, Q. 2010. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering. 22, 10 (Oct. 2010), 1345--1359.
[18]
Schmid, S. et al. 2017. Self-Adaptation Based on Big Data Analytics: A Model Problem and Tool. Proc. of SEAMS 2017 (Buenos Aires, Argentina, May 2017), 102--108.
[19]
Shahriari, B. et al. 2016. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proceedings of the IEEE. 104, 1 (Jan. 2016), 148--175.
[20]
Sheskin, D.J. 2007. Handbook of Parametric and Nonparametric Statistical Procedures. Chapman & Hall/CRC.
[21]
Snoek, J. et al. 2012. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems (2012), 2951-2959.
[22]
Ustinova, T. and Jamshidi, P. 2015. Modelling multi-tier enterprise applications behaviour with design of experiments technique. Proceedings of the 1st International Workshop on Quality-Aware DevOps (2015), 13--18.

Cited By

View all
  • (2024)Doktor nöbet cetveli çizelgeleme problemi için ağırlıklı hedef programlama tabanlı genetik algoritmaGazi Üniversitesi Mühendislik Mimarlık Fakültesi Dergisi10.17341/gazimmfd.135553339:4(2567-2586)Online publication date: 20-May-2024
  • (2022)Does configuration encoding matter in learning software performance?Proceedings of the 19th International Conference on Mining Software Repositories10.1145/3524842.3528431(482-494)Online publication date: 23-May-2022
  • (2022)Rango: An Intuitive Rule Language for Learning Classifier Systems in Cyber-Physical Systems2022 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)10.1109/ACSOS55765.2022.00021(31-40)Online publication date: Sep-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SEAMS '18: Proceedings of the 13th International Conference on Software Engineering for Adaptive and Self-Managing Systems
May 2018
244 pages
ISBN:9781450357159
DOI:10.1145/3194133
  • General Chair:
  • Jesper Andersson,
  • Program Chair:
  • Danny Weyns
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. experimentation cost
  2. self-adaptation
  3. statistical guarantees

Qualifiers

  • Research-article

Funding Sources

  • Ministry of Education, Youth and Sports of the Czech Republic
  • Bayerisches Staatsministerium fur Wirtschaft und Medien, Energie und Technologie
  • German Ministry of Education and Research (BMBF)

Conference

ICSE '18
Sponsor:

Acceptance Rates

Overall Acceptance Rate 17 of 31 submissions, 55%

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Doktor nöbet cetveli çizelgeleme problemi için ağırlıklı hedef programlama tabanlı genetik algoritmaGazi Üniversitesi Mühendislik Mimarlık Fakültesi Dergisi10.17341/gazimmfd.135553339:4(2567-2586)Online publication date: 20-May-2024
  • (2022)Does configuration encoding matter in learning software performance?Proceedings of the 19th International Conference on Mining Software Repositories10.1145/3524842.3528431(482-494)Online publication date: 23-May-2022
  • (2022)Rango: An Intuitive Rule Language for Learning Classifier Systems in Cyber-Physical Systems2022 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)10.1109/ACSOS55765.2022.00021(31-40)Online publication date: Sep-2022
  • (2022)A literature review on optimization techniques for adaptation planning in adaptive systemsInformation and Software Technology10.1016/j.infsof.2022.106940149:COnline publication date: 1-Sep-2022
  • (2022)Proactive hybrid learning and optimisation in self-adaptive systemsInformation and Software Technology10.1016/j.infsof.2022.106826145:COnline publication date: 1-May-2022
  • (2021)Utility-based Vehicle Routing Integrating User Preferences2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops51409.2021.9431066(263-268)Online publication date: 22-Mar-2021
  • (2021)Automated Online Experiment-Driven Adaptation–Mechanics and Cost AspectsIEEE Access10.1109/ACCESS.2021.30718099(58079-58087)Online publication date: 2021
  • (2021)An ICT System to Assist Earth Fault LocalizationDependable Computing - EDCC 2021 Workshops10.1007/978-3-030-86507-8_5(51-64)Online publication date: 8-Sep-2021
  • (2020)A hybrid approach combining control theory and AI for engineering self-adaptive systemsProceedings of the IEEE/ACM 15th International Symposium on Software Engineering for Adaptive and Self-Managing Systems10.1145/3387939.3391595(9-19)Online publication date: 29-Jun-2020
  • (2019)Automated Optimization of Software Parameters in a Long Term Evolution Radio Base Station2019 IEEE International Systems Conference (SysCon)10.1109/SYSCON.2019.8836830(1-8)Online publication date: Apr-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media