Give me some REST: A Controlled Experiment to Study Effects and Perception of Model-Driven Engineering with a Domain-Specific Language
Pages 214 - 225
Abstract
Domain-Specific Languages (DSLs) are an efficient means to counter accidental complexity and are therefore a key technology for Model-Driven Engineering (MDE). Despite DSLs' potential, there is a lack of empirical research regarding the practical effects and developer perception of DSL-driven tools. In this paper, we present a controlled experiment with 28 participants around a previously developed DSL-based toolchain, which assists the migration of legacy software to REST. A direct comparison of developer performance for a) "DSL toolchain" and b) "classic manual software migration" allowed for analysis, quantification of effects and developer perception, as well as reasoning on general advantages, and DSL-related challenges. In certain cases, we measured a significant correlation between toolchain use and performance gains for developers. Detailed analysis of developer activities suggests the DSL toolchain alleviates tasks which show error-prone or time-consuming in the manual alternative. We then extracted acceptance-hindering factors from participant feedback and derived a series of recommendations for MDE practitioners who seek to develop DSL-based tools.
References
[1]
Colin Atkinson and Thomas Kühne. 2008. Reducing accidental complexity in domain models. Software & Systems Modeling 7, 3 (2008), 345--359. https://doi.org/10.1007/s10270-007-0061-0
[2]
Jean-Michel Bruel, Benoit Combemale, Ileana Ober, and Hélène Raynal. 2015. MDE in Practice for Computational Science. Procedia Computer Science 51 (2015), 660--669. https://doi.org/10.1016/j.procs.2015.05.182 International Conference On Computational Science, ICCS 2015.
[3]
Antonio Bucchiarone, Kemal Soysal, and Claudio Guidi. 2020. A Model-Driven Approach Towards Automatic Migration to Microservices. In Software Engineering Aspects of Continuous Development and New Paradigms of Software Production and Deployment, Jean-Michel Bruel, Manuel Mazzara, and Bertrand Meyer (Eds.). Software Engineering Aspects of Continuous Development and New Paradigms of Software Production and Deployment: Second International Workshop, DEVOPS 2019, Vol. 12055. Springer International Publishing, Cham, 15--36. https://doi.org/10.1007/978-3-030-39306-9_2 Series Title: Lecture Notes in Computer Science.
[4]
Cristina Cachero, Santiago Meliá, and Jesús M. Hermida. 2019. Impact of model notations on the productivity of domain modelling: An empirical study. Information and Software Technology 108 (April 2019), 78--87. https://doi.org/10.1016/j.infsof.2018.12.005
[5]
Donald T. Campbell and Julian C. Stanley. 2011. Experimental and quasi-experimental designs for research. Wadsworth, Belomt, CA.
[6]
Mariano Ceccato, Massimiliano Di Penta, Paolo Falcarin, Filippo Ricca, Marco Torchiano, and Paolo Tonella. 2013. A family of experiments to assess the effectiveness and efficiency of source code obfuscation techniques. Empirical Software Engineering (Feb. 2013). https://doi.org/10.1007/s10664-013-9248-x
[7]
Thomas Cook, Donald Campbell, and William Shadish. 2001. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Wadsworth Publishing.
[8]
África Domingo, Jorge Echeverría, Óscar Pastor, and Carlos Cetina. 2021. Comparing UML-Based and DSL-Based Modeling from Subjective and Objective Perspectives. In Advanced Information Systems Engineering, Marcello La Rosa, Shazia Sadiq, and Ernest Teniente (Eds.). Springer International Publishing, Cham, 483--498.
[9]
Nicola Dragoni, Saverio Giallorenzo, Alberto Lluch Lafuente, Manuel Mazzara, Fabrizio Montesi, Ruslan Mustafin, and Larisa Safina. 2017. Microservices: Yesterday, Today, and Tomorrow. In Present and Ulterior Software Engineering, Manuel Mazzara and Bertrand Meyer (Eds.). Springer International Publishing, Cham, 195--216. https://doi.org/10.1007/978-3-319-67425-4_12
[10]
Robert Feldt, Thomas Zimmermann, Gunnar R. Bergersen, Davide Falessi, Andreas Jedlitschka, Natalia Juristo, Jürgen Münch, Markku Oivo, Per Runeson, Martin Shepperd, Dag I. K. Sjøberg, and Burak Turhan. 2018. Four commentaries on the use of students and professionals in empirical software engineering experiments. Empirical Software Engineering 23, 6 (Dec. 2018), 3801--3820. https://doi.org/10.1007/s10664-018-9655-0
[11]
Roy Thomas Fielding. 2000. Architectural styles and the design of network - based software architectures. Ph.D. University of California, Irvine, United States - California. https://www.proquest.com/docview/304591392/abstract/D42F954B21D64B61PQ/1 ISBN: 9780599871182.
[12]
Roy T. Fielding, Richard N. Taylor, Justin R. Erenkrantz, Michael M. Gorlick, Jim Whitehead, Rohit Khare, and Peyman Oreizy. 2017. Reflections on the REST architectural style and "principled design of the modern web architecture" (impact paper award). In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 4--14. https://doi.org/10.1145/3106237.3121282
[13]
Marios Fokaefs and Eleni Stroulia. 2015. Using WADL specifications to develop and maintain REST client applications. In 2015 IEEE International Conference on Web Services. IEEE, Piscataway, NJ, USA, 81--88.
[14]
M. Fowler. 2010. Domain-Specific Languages. Pearson Education. https://books.google.ca/books?id=ri1muolw_YwC
[15]
Mahdi Fahmideh Gholami, Farhad Daneshgar, Ghassan Beydoun, and Fethi Rabhi. 2017. Challenges in migrating legacy software systems to the cloud --- an empirical study. Information Systems 67 (July 2017), 100--113. https://doi.org/10.1016/j.is.2017.03.008
[16]
Jeff Gray. 2007. Domain-Specific Modeling. Handbook of dynamic system modeling 7, Handbook of dynamic system modeling (2007). http://scholar.googleusercontent.com/scholar?q=cache:Ss4EaIQUb80J:scholar.google.com/+Jeff+Gray,+Juha-Pekka+Tolvanen, +Steven+Kelly,+Aniruddha+Gokhale,+Sandeep+Neema, +and+Jonathan+Sprinkle.+2007.+Domain- Specific+Modeling&hl=en&as_sdt=0,5
[17]
Jeff Gray, Kathleen Fisher, Charles Consel, Gabor Karsai, Marjan Mernik, and Juha-Pekka Tolvanen. 2008. DSLs: The Good, the Bad, and the Ugly. In Companion to the 23rd ACM SIGPLAN Conference on Object-Oriented Programming Systems Languages and Applications (Nashville, TN, USA) (OOPSLA Companion '08). Association for Computing Machinery, New York, NY, USA, 791--794. https://doi.org/10.1145/1449814.1449863
[18]
John Hutchinson, Jon Whittle, and Mark Rouncefield. 2014. Model-driven engineering practices in industry: Social, organizational and managerial factors that lead to success or failure. Science of Computer Programming 89 (Sept. 2014), 144--161. https://doi.org/10.1016/j.scico.2013.03.017
[19]
John Hutchinson, Jon Wittle, Mark Rouncefield, and Steinar Kristoffersen. 2011. Empirical Assessment of MDE in Industry (33rd International Conference on Software Engineering (ICSE 2011)). IEEE, Piscataway, NJ.
[20]
A. Jedlitschka and D. Pfahl. 2005. Reporting guidelines for controlled experiments in software engineering. In 2005 International Symposium on Empirical Software Engineering, 2005. IEEE, Noosa Heads, QLD, Australia, 10 pp. https://doi.org/10.1109/ISESE.2005.1541818
[21]
Stephen H. Kan. 2003. Metrics and Models in Software Quality Engineering (second edition ed.). Addison Wesley.
[22]
Richard B. Kieburtz, Laura McKinney, Jeffrey M. Bell, James Hook, Alex Kotov, Jeffrey Lewis, Dino P. Oliva, Tim Sheard, Ira Smith, and Lisa Walton. 1996. A Software Engineering Experiment in Software Component Generation. In Proceedings of the 18th International Conference on Software Engineering (Berlin, Germany) (ICSE '96). IEEE Computer Society, USA, 542--552.
[23]
Jörg Kienzle. 2023. TouchCORE. http://touchcore.cs.mcgill.ca/
[24]
Jörg Kienzle, Gunter Mussbacher, Benoit Combemale, and Julien Deantoni. 2019. A unifying framework for homogeneous model composition. Software & Systems Modeling 18, 5 (03 Jan 2019), 3005--3023. https://doi.org/10.1007/s10270-018-00707-8
[25]
Amy J. Ko, Thomas D. LaToza, and Margaret M. Burnett. 2015. A practical guide to controlled experiments of software engineering tools with human participants. Empir. Softw. Eng. 20, 1 (2015), 110--141. https://doi.org/10.1007/s10664-013-9279-3
[26]
Tomaž Kosar, Sašo Gaberc, Jeffrey C. Carver, and Marjan Mernik. 2018. Program comprehension of domain-specific and general-purpose languages: replication of a family of experiments using integrated development environments. Empirical Software Engineering 23, 5 (2018), 2734--2763. https://doi.org/10.1007/s10664-017-9593-2
[27]
Tomaž Kosar, Marjan Mernik, and Jeffrey C. Carver. 2012. Program comprehension of domain-specific and general-purpose languages: comparison using a family of experiments. Empirical Software Engineering 17, 3 (June 2012), 276--304. https://doi.org/10.1007/s10664-011-9172-x
[28]
Simon Maple and Andrew Binstock. 2018. JVM Ecosystem Report 2018 - About your Platform and Application. https://snyk.io/blog/jvm-ecosystem-report-2018-platform-application/ https://snyk.io/blog/jvm-ecosystem-report-2018-platform-application/.
[29]
Yulkeidi Martínez, Cristina Cachero, and Santiago Meliá. 2014. Empirical study on the maintainability of Web applications: Model-driven Engineering vs Code-centric. Empirical Software Engineering 19, 6 (Dec. 2014), 1887--1920. https://doi.org/10.1007/s10664-013-9269-5
[30]
Mark Masse. 2011. REST API Design Rulebook. O'Reilly Media. https://www.oreilly.com/library/view/rest-api-design/9781449317904/
[31]
Andreas Menychtas, Christina Santzaridou, George Kousiouris, Theodora Varvarigou, Leire Orue-Echevarria, Juncal Alonso, Jesus Gorronogoitia, Hugo Bruneliere, Oliver Strauss, Tatiana Senkova, Bram Pellens, and Peter Stuer. 2013. ARTIST Methodology and Framework: A Novel Approach for the Migration of Legacy Software on the Cloud. In 2013 15th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing. IEEE, Timisoara, Romania, 424--431. https://doi.org/10.1109/SYNASC.2013.62
[32]
Mulesoft. 2023. RAML Language Definition. https://github.com/raml-org/ramlspec
[33]
Richard F. Paige and Dániel Varró. 2012. Lessons learned from building modeldriven development tools. Software & Systems Modeling 11, 4 (Oct. 2012), 527--539. https://doi.org/10.1007/s10270-012-0257-9
[34]
API Platform Postman. 2023. 2023 State of the API Report | API Technologies. https://www.postman.com/state-of-api/api-technologies/https://www.postman.com/state-of-api/api-technologies/.
[35]
Kate Revoredo, Djordje Djurica, and Jan Mendling. 2021. A study into the practice of reporting software engineering experiments. Empirical Software Engineering 26, 6 (Nov. 2021), 113. https://doi.org/10.1007/s10664-021-10007-3
[36]
Roberto Rodríguez-Echeverría, Fernando Maclas, Vlctor M. Pavón, José M. Conejero, and Fernando Sánchez-Figueroa. 2014. Generating a REST Service Layer from a Legacy System. In Information System Development, María José Escalona, Gustavo Aragón, Henry Linger, Michael Lang, Chris Barry, and Christoph Schneider (Eds.). Springer International Publishing, Cham, 433--444. https://doi.org/10.1007/978-3-319-07215-9_35
[37]
Maximilian Schiedermeier. 2024. Artifacts for RESTify Experiment. Data, custom tools, and material used and collected throughout the experiment. https://doi.org/10.5281/zenodo.12555385 Publisher: Zenodo.
[38]
Maximilian Schiedermeier, Jörg Kienzle, and Bettina Kemme. 2021. FIDDLR: streamlining reuse with concern-specific modelling languages. In Proceedings of the 14th ACM SIGPLAN International Conference on Software Language Engineering (SLE 2021). Association for Computing Machinery, New York, NY, USA, 164--176. https://doi.org/10.1145/3486608.3486913
[39]
S M Sohan, Frank Maurer, Craig Anslow, and Martin P. Robillard. 2017. A study of the effectiveness of usage examples in REST API documentation. In 2017 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, Raleigh, NC, 53--61. https://doi.org/10.1109/VLHCC.2017.8103450
[40]
Klaas-Jan Stol and Brian Fitzgerald. 2018. The ABC of Software Engineering Research. ACM Transactions on Software Engineering and Methodology 27, 3 (July 2018), 1--51. https://doi.org/10.1145/3241743
[41]
Swagger. 2023. API Code & Client Generator | Swagger Codegen. https://swagger.io/tools/swagger-codegen/ https://swagger.io/tools/swagger-codegen/.
[42]
Arie van Deursen, Paul Klint, and Joost Visser. 2000. Domain-Specific Languages: An Annotated Bibliography. SIGPLAN Not. 35, 6 (jun 2000), 26--36. https://doi.org/10.1145/352029.352035
[43]
Sira Vegas, Cecilia Apa, and Natalia Juristo. 2016. Crossover Designs in Software Engineering Experiments: Benefits and Perils. IEEE Transactions on Software Engineering 42, 2 (Feb. 2016), 120--135. https://doi.org/10.1109/TSE.2015.2467378 Conference Name: IEEE Transactions on Software Engineering.
[44]
Phillip Webb, Dave Syer, Josh Long, Stéphane Nicoll, Rob Winch, Andy Wilkinson, Marcel Overdijk, Christian Dupuis, Sébastien Deleuze, Michael Simons, Vedran Pavić, Jay Bryant, and Madhura Bhave. 2018. Spring Boot Reference Guide. (2018).
[45]
Jon Whittle, John Hutchinson, Mark Rouncefield, Håkan Burden, and Rogardt Heldal. 2013. Industrial Adoption of Model-Driven Engineering: Are the Tools Really the Problem? In Model-Driven Engineering Languages and Systems, David Hutchison, Takeo Kanade, Josef Kittler, Jon M. Kleinberg, Friedemann Mattern, John C. Mitchell, Moni Naor, Oscar Nierstrasz, C. Pandu Rangan, Bernhard Steffen, Madhu Sudan, Demetri Terzopoulos, Doug Tygar, Moshe Y. Vardi, Gerhard Weikum, Ana Moreira, Bernhard Schätz, Jeff Gray, Antonio Vallecillo, and Peter Clarke (Eds.). Vol. 8107. Springer Berlin Heidelberg, Berlin, Heidelberg, 1--17. https://doi.org/10.1007/978-3-642-41533-3_1 Series Title: Lecture Notes in Computer Science.
[46]
Jon Whittle, John Hutchinson, Mark Rouncefield, Håkan Burden, and Rogardt Heldal. 2017. A taxonomy of tool-related issues affecting the adoption of modeldriven engineering. Software & Systems Modeling 16, 2 (May 2017), 313--331. https://doi.org/10.1007/s10270-015-0487-8
[47]
D. Wile. 2003. Lessons learned from real DSL experiments. In 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the. 10 pp.-. https://doi.org/10.1109/HICSS.2003.1174893
[48]
Claes Wohlin, Martin Höst, and Kennet Henningsson. 2003. Empirical Research Methods in Software Engineering. In Empirical Methods and Studies in Software Engineering, Gerhard Goos, Juris Hartmanis, Jan Van Leeuwen, Reidar Conradi, and Alf Inge Wang (Eds.). Vol. 2765. Springer Berlin Heidelberg, Berlin, Heidelberg, 7--23. https://doi.org/10.1007/978-3-540-45143-3_2 Series Title: Lecture Notes in Computer Science.
[49]
Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in Software Engineering. Springer Berlin Heidelberg, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29044-2
Recommendations
Model-driven engineering with domain-specific meta-modelling languages
Domain-specific modelling languages are normally defined through general-purpose meta-modelling languages like the MOF. While this is satisfactory for many model-driven engineering (MDE) projects, several researchers have identified the need for domain-...
Leveraging model driven engineering in software product line architectures
SPLC '14: Proceedings of the 18th International Software Product Line Conference - Volume 1The process of Developing Software Product Line Architectures can be a complex task. However, the use of Model Driven Engineering (MDE) techniques can facilitate the development of SPLAs by introducing Domain Specific Languages, Graphical Editors, and ...
Leveraging model driven engineering in software product line architectures
SPLC '15: Proceedings of the 19th International Conference on Software Product LineThe process of Developing Software Product Line Architectures can be a complex task. However, the use of Model Driven Engineering (MDE) techniques can facilitate the development of SPLAs by introducing Domain Specific Languages, Graphical Editors, and ...
Comments
Information & Contributors
Information
Published In
September 2024
311 pages
ISBN:9798400705045
DOI:10.1145/3640310
- General Chairs:
- Alexander Egyed,
- Manuel Wimmer,
- Program Chairs:
- Marsha Chechik,
- Benoit Combemale
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
- Johannes Kepler University, Linz, Austria
- SIGSOFT: ACM Special Interest Group on Software Engineering
- IEEE CS
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 September 2024
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
MODELS '24
Sponsor:
MODELS '24: ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems
September 22 - 27, 2024
Linz, Austria
Acceptance Rates
MODELS '24 Paper Acceptance Rate 26 of 124 submissions, 21%;
Overall Acceptance Rate 144 of 506 submissions, 28%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 62Total Downloads
- Downloads (Last 12 months)62
- Downloads (Last 6 weeks)12
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in