skip to main content
10.1145/3417990.3421395acmconferencesArticle/Chapter ViewAbstractPublication PagesmodelsConference Proceedingsconference-collections
research-article

A comparative study of reinforcement learning techniques to repair models

Published: 26 October 2020 Publication History

Abstract

In model-driven software engineering, models are used in all phases of the development process. These models may get broken due to various editions during the modeling process. To repair broken models we have developed PARMOREL, an extensible framework that uses reinforcement learning techniques. So far, we have used our version of the Markov Decision Process (MDP) adapted to the model repair problem and the Q-learning algorithm. In this paper, we revisit our MDP definition, addressing its weaknesses, and proposing a new one. After comparing the results of both MDPs using Q-Learning to repair a sample model, we proceed to compare the performance of Q-Learning with other reinforcement learning algorithms using the new MDP. We compare Q-Learning with four algorithms: Q(λ), Monte Carlo, SARSA and SARSA (λ), and perform a comparative study by repairing a set of broken models. Our results indicate that the new MDP definition and the Q(λ) algorithm can repair with faster performance.

References

[1]
Angela Barriga, Adrian Rutle, and Heldal Rogardt. 2020. Improving model repair through experience sharing. Journal of Object Technology 19, 2 (July 2020), 13:1--21.
[2]
Francesco Basciani, Juri Di Rocco, Davide Di Ruscio, Ludovico Iovino, and Alfonso Pierantonio. 2016. A customizable approach for the automated quality assessment of modelling artifacts. In 2016 10th International Conference on the Quality of Information and Communications Technology (QUATIC). IEEE, 88--93.
[3]
Richard Bellman. 2013. Dynamic programming. Courier Corporation.
[4]
Marcela Genero and Mario Piattini. 2001. Empirical validation of measures for class diagram structural complexity through controlled experiments. In 5th International ECOOP Workshop on Quantitative Approaches in Object-Oriented Software Engineering.
[5]
Ludovico Iovino, Angela Barriga, Adrian Rutle, and Heldal Rogardt. 2020. Model Repair with Quality-Based Reinforcement Learning. Journal of Object Technology 19, 2 (July 2020), 17:1--21.
[6]
Marouane Kessentini, Usman Mansoor, Manuel Wimmer, Ali Ouni, and Kalyanmoy Deb. 2017. Search-based detection of model level changes. Empirical Software Engineering 22, 2 (2017), 670--715.
[7]
Roland Kretschmer, Djamel Eddine Khelladi, and Alexander Egyed. 2018. An automated and instant discovery of concrete repairs for model inconsistencies. In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings. ACM, 298--299.
[8]
Nuno Macedo, Tiago Guimaraes, and Alcino Cunha. 2013. Model repair and transformation with Echo. In Proceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering. IEEE Press, 694--697.
[9]
Lawrence Mandow and José-Luis Pérez-de-la Cruz. 2018. Pruning Dominated Policies in Multiobjective Pareto Q-Learning. In Conference of the Spanish Association for Artificial Intelligence. Springer, 240--250.
[10]
Iman Hemati Moghadam and Mel Ó Cinnéide. 2011. Code-Imp: a tool for automated search-based refactoring. In Proceedings of the 4th Workshop on Refactoring Tools. 41--44.
[11]
Martin Mundhenk, Judy Goldsmith, Christopher Lusena, and Eric Allender. 2000. Complexity of finite-horizon Markov decision process problems. Journal of the ACM (JACM) 47, 4 (2000), 681--720.
[12]
Nebras Nassar, Hendrik Radke, and Thorsten Arendt. 2017. Rule-Based Repair of EMF Models: An Automated Interactive Approach. In International Conference on Theory and Practice of Model Transformations. Springer, 171--181.
[13]
Phuong T Nguyen, Juri Di Rocco, Davide Di Ruscio, Alfonso Pierantonio, and Ludovico Iovino. 2019. Automated Classification of Metamodel Repositories: A Machine Learning Approach. In 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS). IEEE, 272--282.
[14]
Manuel Ohrndorf, Christopher Pietsch, Udo Kelter, and Timo Kehrer. 2018. ReVision: a tool for history-based model repair recommendations. In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings. ACM, 105--108.
[15]
Jorge Pinna Puissant, Ragnhild Van Der Straeten, and Tom Mens. 2015. Resolving model inconsistencies using automated regression planning. Software & Systems Modeling 14, 1 (2015), 461--481.
[16]
Alexander Reder and Alexander Egyed. 2012. Computing repair trees for resolving inconsistencies in design models. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering. ACM, 220--229.
[17]
Bart Selman and Carla P Gomes. 2006. Hill-climbing Search. Encyclopedia of cognitive science (2006).
[18]
Dave Steinberg, Frank Budinsky, Ed Merks, and Marcelo Paternostro. 2008. EMF: eclipse modeling framework. Pearson Education.
[19]
Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.
[20]
Gabriele Taentzer, Manuel Ohrndorf, Yngve Lamo, and Adrian Rutle. 2017. Change-preserving model repair. In International Conference on Fundamental Approaches to Software Engineering. Springer, 283--299.
[21]
Jon Whittle, John Hutchinson, and Mark Rouncefield. 2014. The state of practice in model-driven engineering. IEEE software 31, 3 (2014), 79--85.
[22]
Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in software engineering. Springer Science & Business Media.

Cited By

View all
  • (2025)Can explainable artificial intelligence support software modelers in model comprehension?Software and Systems Modeling10.1007/s10270-024-01251-4Online publication date: 2-Jan-2025
  • (2024)From single-objective to multi-objective reinforcement learning-based model transformationSoftware and Systems Modeling10.1007/s10270-024-01233-6Online publication date: 19-Nov-2024
  • (2024)Improving repair of semantic ATL errors using a social diversity metricSoftware and Systems Modeling (SoSyM)10.1007/s10270-024-01170-423:6(1547-1568)Online publication date: 1-Dec-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MODELS '20: Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings
October 2020
713 pages
ISBN:9781450381352
DOI:10.1145/3417990
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

  • IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. markov decision process
  2. model repair
  3. reinforcement learning

Qualifiers

  • Research-article

Funding Sources

  • Spanish Government, Agencia Estatal de Investigacion (AEI)
  • European Union, Fondo Europeo de Desarrollo Regional (FEDER)

Conference

MODELS '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 144 of 506 submissions, 28%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)32
  • Downloads (Last 6 weeks)1
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Can explainable artificial intelligence support software modelers in model comprehension?Software and Systems Modeling10.1007/s10270-024-01251-4Online publication date: 2-Jan-2025
  • (2024)From single-objective to multi-objective reinforcement learning-based model transformationSoftware and Systems Modeling10.1007/s10270-024-01233-6Online publication date: 19-Nov-2024
  • (2024)Improving repair of semantic ATL errors using a social diversity metricSoftware and Systems Modeling (SoSyM)10.1007/s10270-024-01170-423:6(1547-1568)Online publication date: 1-Dec-2024
  • (2023)Model-Driven Optimization for Quantum Program Synthesis with MOMoT2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00100(614-621)Online publication date: 1-Oct-2023
  • (2022)Two-Level Optimization to Reduce Waiting Time at Locks in Inland Waterway TransportationACM Transactions on Intelligent Systems and Technology10.1145/352782213:6(1-30)Online publication date: 22-Apr-2022
  • (2022)PARMOREL: a framework for customizable model repairSoftware and Systems Modeling10.1007/s10270-022-01005-021:5(1739-1762)Online publication date: 4-May-2022
  • (2021)Towards Reinforcement Learning for In-Place Model Transformations2021 ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS)10.1109/MODELS50736.2021.00017(82-88)Online publication date: Oct-2021
  • (2021)Automated Patch Generation for Fixing Semantic Errors in ATL Transformation Rules2021 ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS)10.1109/MODELS50736.2021.00011(13-23)Online publication date: Oct-2021
  • (2021)Comprehensive Systems: A formal foundation for Multi-Model Consistency ManagementFormal Aspects of Computing10.1007/s00165-021-00555-233:6(1067-1114)Online publication date: 30-Jul-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media