Skip to main content
SpringerLink
Log in

How consistency is handled in model-driven software engineering and UML: an expert opinion survey

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Model-driven software engineering (MDSE) is an established approach for developing complex software systems. The unified modelling language (UML) is one of the most used model languages for applying the MDSE approach. UML has 14 diagram types that describe different perspectives of a software system under development. These diagrams are strongly dependent on each other and must be consistent with one another. The main objectives of this paper are as follows: (1) to understand (i) how aware experts are of model consistency issues and (ii) how relevant these issues are to experts, in order to understand model consistency in the MDSE/UML contexts, and more importantly, (2) to validate a set of 116 UML consistency rules that was systematically collected from the literature, so as to identify the rules that should always be enforced. We conducted a personal opinion survey with 106 experts in SE and MDSE, by means of an online questionnaire. The survey results describe an overview of how the topic of MDSE/UML consistency is handled by experts in the field. In addition, this survey identified a set of 52 UML consistency rules which should always be checked in every UML diagram. The majority of these 52 rules were understood by the majority of respondents and are general-purpose rules that are involved in the Design software development phase. This subset of 52 rules could be considered to be (1) added to the UML standard, (2) used as a reference to researchers who study UML/MDSE, and (3) used as a practical example for teaching purposes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. https://shorturl.at/efo47

  2. https://wucor.wordpress.com

  3. http://cruise.eecs.uottawa.ca/models2015/

  4. https://sselab.de/lab2/public/wiki/MiSE/index.php?title=MISE_16_-_Austin,_USA

  5. https://bit.ly/2xk68Ic

  6. https://shorturl.at/efo47

References 

  • Ahmad, M. A., & Nadeem, A. (2010). Consistency checking of UML models using description logics: a critical review. Paper presented at the 6th International Conference on Emerging Technologies Islamabad, Pakistan, October 18–19

  • Allaki, D., Dahchour, M., & En-Nouaary, A. (2017). Managing inconsistencies in UML models: a systematic literature review. Journal of Software, 12(6), 454–471.

    Article  Google Scholar 

  • Akdur, D., Demirörs, O., & Garousi, V. (2015) The results of a world-wide survey on software modeling and model-driven engineering in the embedded system domain Technical Report METU/II-2015-55

  • Arisholm, E., Briand, L. C., & Anda, B. C. D. (2008). First workshop on empirical studies of model-driven engineering at MODELS 2008.

  • Arlow, J., & Neustadt, I. (2005). UML 2.0 and the Unified Process: practical object-oriented analysis and design, 2nd ed. Addison-Wesley Professional

  • Brambilla, M., Cabot, J., & Wimmer, M. (2016). Model-driven software engineering in practice. 2nd edn.

  • Briand, L. C., Labiche, Y., & O'Sullivan, L. (2003). Impact analysis and change management of UML models. Paper presented at the International Conference on Software Maintenance, Amsterdam, The Netherlands, September 2003

  • Burgueño, L., Ciccozzi, F., Famelis, M., Kappel, G., Lambers, L., Mosser, S., et al. (2019). Contents for a model-based software engineering body of knowledge. Software and Systems Modeling, 18, 3193–3205.

    Article  Google Scholar 

  • Cicchetti, A., Ciccozzi, F., & Pierantonio, A. (2019). Multi-view approaches for software and system modelling: a systematic literature review. Software and Systems Modeling, 18, 3207–3233.

    Article  Google Scholar 

  • Dobing, B., & Parsons, J. (2006). How UML is used ACM, 49, 109–113. https://doi.org/10.1145/1125944.1125949

    Article  Google Scholar 

  • Dragomir, I., Graf, S., Karsai, G., Noyrit, F., Ober, I., Torre, D., Labiche, Y., Genero, M., Elaasar, M. (2015). Joint Proceedings of the 8th International Workshop on Model-based Architecting of Cyber-physical and Embedded Systems and 1st International Workshop on UML Consistency Rules (ACES-MB 2015 & WUCOR 2015) co-located with ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MoDELS 2015), Ottawa, Canada, September 28, 2015. CEUR Workshop Proceedings 1508, CEUR-WS.org 2015.

  • Engels, G., Hausmann, J. H., & Heckel, R. (2002). Testing the consistency of dynamic UML diagrams. Paper presented at the Integrated Design and Process Technology, Pasadena, California, June 24.

  • Engels, G., Heckel, R., & Küster, J. M. (2001a). Rule-based specification of behavioral consistency based on the UML meta-model. Paper presented at the 4th International Conference on The Unified Modeling Language, Modeling Languages, Concepts, and Tools, Toronto, Ontario, Canada, October 1–5.

  • Engels, G., Küster, J. M., Heckel, R., & Groenewegen, L. (2001b). A methodology for specifying and analyzing consistency of object-oriented behavioral models. Sigsoft Software Engineering. Notes, 26, 186–195. https://doi.org/10.1145/503209.503235

    Article  Google Scholar 

  • Fernández-Sáez, A. M., Genero, M., Caivano, D., & Chaudron, M. R. V. (2015). On the use of UML documentation in software maintenance: results from a survey in industry. Paper presented at the 18th International Conference on Model Driven Engineering Languages and Systems (MODELS), 2015 ACM/IEEE.

  • Forward, A., Badreddin, O., & Lethbridge, T. C. (2010). Perceptions of software modeling: a survey of software practitioners. Paper presented at the 5th Workshop From code centric to model centric: Evaluating the effectiveness of MDD (C2M:EEMDD), Paris, France.

  • Frankel, D. (2003). Model Driven Architecture: Applying MDA to enterprise computing. John Wiley & Sons.

  • Genero, M., Fernández-Saez, A. M., Nelson, H. J., Poels, G., & Piattini, M. (2011). A systematic literature review on the quality of UML models. Journal of Database Management, 22, 46–70. https://doi.org/10.4018/jdm.2011070103

    Article  Google Scholar 

  • Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology. Second edn. John Wiley & Sons.

  • Gutbrod, R., & Wiele, C. (2012). The software dilemma balancing creativity and control on the path to sustainable software. Springer.

    Book  Google Scholar 

  • Hunt, J. (2003). Guide to the Unified Process featuring UML, Java and design patterns (2nd ed.). Springer-Verlag.

    Google Scholar 

  • Hutchinson, J., Whittle, J., & Rouncefield, M. (2014). Model-driven engineering practices in industry: Social, organizational and managerial factors that lead to success or failure. Science of Computer Programming, 89, 144–161

  • Huzar, Z., Kuzniarz, L., Reggio, G., & Sourrouille, J. L. (2005). Consistency problems in UML-based software development. Paper presented at the International Conference on UML Modeling Languages and Applications, Lisbon, Portugal.

  • Ibrahim, N., Ibrahim, R., Saringat, M. Z., Mansor, D., & Herawan, T. (2011). Consistency rules between UML use case and activity diagrams using logical approach International Journal of Software Engineering and its Applications, 5, 119–134.

    Google Scholar 

  • IEEE. (1992). IEEE standard for software veriÞcation and validation plans (IEEE Std 1012-1986). https://standards.ieee.org/findstds/standard/1012-1986.html (last accessed on May 2018).

  • Jedlitschka, A., Ciolkowski, M., Denger, C., Freimut, B., & Schlichting, A. (2007). Relevant information sources for successful technology transfer: a survey using inspections as an example. Paper presented at the the International Symposium on Empirical Software Engineering and Measurement (ESEM).

  • Kalibatiene, D., Vasilecas, O., & Dubauskaite, R. (2013). Rule based approach for ensuring consistency in different UML models. Paper presented at the 6th SIGSAND/PLAIS EuroSymposium 2013, Gdańsk, Poland, September 26.

  • Kitchenham, B., Budgen, D., & Brereton, P. (2015). Evidence-based software engineering and systematic reviews. Chapman & Hall/CRC.

  • Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Keele University.

  • Kitchenham, B. A., & Pfleeger, S. L. (2008). Personal opinion surveys. guide to advanced empirical software engineering (pp. 63–92). Springer.

    Chapter  Google Scholar 

  • Knapp, A., & Mossakowski, T. (2018). Multi-view Consistency in UML: A Survey. In: Heckel R., Taentzer G. (eds) Graph transformation, specifications, and nets. Lecture Notes in Computer Science, 10800. Springer.

  • Krosnick, J. A. (1990). Survey research. Annual Review of Psychology, 50, 537–567.

    Article  Google Scholar 

  • Labiche, Y. (2008). The UML is more than boxes and lines. Paper presented at the Workshops and Symposia at MODELS 2008, Toulouse, France, September 28 - October 3.

  • Lethbridge, T., Cabot, J., Egyed, A. (2015). 18th ACM/IEEE International Conference on Model Driven Engineering Languages and Systems, MoDELS 2015, Ottawa, ON, Canada, September 30 - October 2, 2015. IEEE Computer Society 2015, ISBN 978-1-4673-6908-4.

  • Linaker, J., Sulaman, S. M., Mello, R. M. D., & Höst, M. (2015). Guidelines for conducting surveys in software engineering, TR 5366801. Lund University.

  • Lucas, F. J., Molina, F., & Toval, A. (2009). A systematic review of UML model consistency management. Information and Software Technology, 51, 1631–1645. https://doi.org/10.1016/j.infsof.2009.04.009

    Article  Google Scholar 

  • Maxwell, J. A. (2010). Using numbers in qualitative research Qualitative Inquiry, 16, 475–482.

    Article  Google Scholar 

  • Mens, T., Van der Straeten, R., & Simmonds, J. (2005). A framework for managing consistency of evolving UML models. In: Software evolution with UML and XML. IGI Publishing, pp 1–30.

  • Mukerji, J., & Miller, J. (2003). Overview and guide to OMG's architecture. Object Management Group.

  • Muskens, J., Bril, R. J., & Chaudron, M. R. V. (2005). Generalizing consistency checking between software views. Paper presented at the 5th Working IEEE/IFIP Conference on Software Architecture, Pittsburgh, Pennsylvania, USA, November 6–10.

  • NATO. (1968). Software engineering. Paper presented at the Report on a conference sponsored by the NATO SCIENCE COMMITTEE, Garmisch, Germany.

  • OMG. (2015). OMG Unified Modeling LanguageTM - Superstructure Version 2.5. https://www.omg.org/spec/UML/2.5/About-UML/ (last accessed on May 2018).

  • OMG. (2016). Object Management Group - Object Constraint Language (OCL).

  • Paige, R. F., Kolovos, D. S., & Polack, F. A. C. (2005). Refinement via consistency checking in MDA. Electronic Notes in Theoretical Computer Science, 137, 151–161. https://doi.org/10.1016/j.entcs.2005.04.029

    Article  MATH  Google Scholar 

  • Panesar-Walawege, R., Sabetzadeh, M., & Briand, L. (2013). Supporting the verification of compliance to safety standards via model-driven engineering: Approach, tool-support and empirical validation. Informationand Software Technology, 55(5), 836–886.

  • Pap, Z., Majzik, I., Pataricza, A., & Szegi, A. (2005). Methods of checking general safety criteria in UML statechart specifications. Reliability Engineering & System Safety, 87, 89–107. https://doi.org/10.1016/j.ress.2004.04.011

    Article  Google Scholar 

  • Pender, T. (2003). UML bible (1st ed.). John Wiley & Sons Inc.

    Google Scholar 

  • Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information & Software Technology, 64, 1–18. https://doi.org/10.1016/j.infsof.2015.03.007

    Article  Google Scholar 

  • Petre, M. (2013). UML in practice. Paper presented at the 35th International Conference on Software Engineering, San Francisco, CA, USA, May 18th–26th.

  • Pilskalns, O., Williams, D., Aracic, D., & Andrews, A. (2006). Security consistency in UML designs. Paper presented at the 30th Annual International Computer Software and Applications Conference, Chicago, USA, September 17–21.

  • Pinsonneault, A., & Kraemer, K. L. (1993). Survey research methodology in management information systems: an assessment Journal of Management Information Systems - Special section: Strategic and competitive information systems, 10, 75–105.

    Google Scholar 

  • Pinsonneault, A., & Kraemer, K. L. (1993). Survey research methodology in management information systems: an assessment. Journal of Management Information Systems, 10, 75–105.

    Article  Google Scholar 

  • Punter, T., Ciolkowski, M., Freimut, B., & John, I. (2003a). Conducting on-line surveys in software engineering. Paper presented at the International Symposium on Empirical Software Engineering.

  • Punter, T., Ciolkowski, M., Freimut, B., & John, I. (2003b). Conducting on-line surveys in software engineering. Paper presented at the the International Symposium on Empirical Software Engineering (ISESE).

  • Qu, H., & Veres, S. (2016). Verification of logical consistency inrobotic reasoning. Robotics and Autonomous Systems, 83, 44–56.

  • Rumbaugh, J., Jacobson, I., & Booch, G. (2004). Unified Modeling Language reference manual, 2nd ed. Pearson Higher Education.

  • Reggio, G., Leotta, M., Ricca, F., & Clerissi, D. (2013). What are the used UML diagrams? A preliminary survey. Paper presented at the In Proceedings of 3rd International Workshop on Experiences and Empirical Studies in Software Modeling - CEUR Workshop Proceedings (EESSMod 2013), Miami, Florida - USA, October 1.

  • Rodríguez, P., Markkula, J., Oivo, M., & Turula, K. (2012). Survey on agile and lean usage in finnish software industry. Paper presented at the the ACM-IEEE international symposium on Empirical software engineering and measurement (ESEM '12).

  • Scanniello, G., Gravino, C., & Tortora, G. (2010). Investigating the role of UML in the software modeling and maintenance - A preliminary industrial survey. Paper presented at the the 12th International Conference on Enterprise Information Systems (ICEIS'2010), Funchal, Madeira, Portugal.

  • Simmonds, J., Straeten, R. V., Jonkers, V., & Mens, T. (2004). Maintaining consistency between UML models using description logicZ RSTI – L’Object LMO’04 10, 231–244. https://doi.org/10.3166/objet.10.2-3.231-244

  • Song, I-Y., Khare, R., An, Y., & Hilsbos, M. (2008). A multi-level methodology for developing UML sequence diagrams. Paper presented at the 27th International Conference on Conceptual Modeling, Barcelona, Spain, October 20–24.

  • Spanoudakis, G., & Zisman, A. (2001). Inconsistency management in software engineering: survey and open research issues. In S. K. Chang (Ed.), Handbook of Software Engineering and Knowledge Engineering (pp. 329–380). World Scientific Publishing Co.

    Chapter  Google Scholar 

  • Survey Monkey. (1999) https://www.surveymonkey.net/. Accessed last access on November 2016.

  • Telinski, L., Agner, W., Soares, I. W., Stadzisz, P. C., & Simão, J. M. (2013). A Brazilian survey on UML and model-driven practices for embedded software development J Syst Softw, 86, 997–1005.

    Google Scholar 

  • Thomas, D. (2004). MDA: Revenge of the modelers or UML utopia? IEEE Software, 21, 15–17.

    Article  Google Scholar 

  • Torre, D. (2014). On collecting and validating UML consistency rules: a research proposal. Paper presented at the Doctoral Symposium at 18th International Conference on Evaluation and Assessment in Software Engineering (EASE 2014), London, UK, 12 May.

  • Torre, D. (2016). Verifying the consistency of UML models. Paper presented at the 27th IEEE International Symposium on Software Reliability Engineering, Fast Abstract (ISSRE 2016) Ottawa, Canada.

  • Torre, D., Labiche, Y., & Genero, M. (2014). UML consistency rules: a systematic mapping study. Paper presented at the 18th International Conference on Evaluation and Assessment in Software Engineering (EASE 2014), London, UK, 13-14 May.

  • Torre, D., Labiche, Y., Genero, M., & Elaasar, M. (2018). A systematic identification of consistency rules for UML diagrams. Journal of Systems and Software, 144, 121–142.

    Article  Google Scholar 

  • Torre, D., Labiche, Y., Genero, M., Elaasar, M., Das, T. K., Hoisl, B., & Kowal, M. (2016). 1st International Workshop on UML Consistency Rules (WUCOR) Post workshop report SIGSOFT. Software Engineering Notes, 41, 34–37. https://doi.org/10.1145/2894784.2894801

    Article  Google Scholar 

  • Torre, D., Soltana, G., Sabetzadeh, M., Briand, L., Auffinger, Y., & Goes, P. (2019). Using models to enable compliance checking against the GDPR: An experience report. 22nd ACM/IEEE InternationalConference on Model Driven Engineering Languages and Systems, (MODELS 2019) Munich, Germany.

  • Torre, D., Labiche, Y., Genero, M., Elaasar, M., & Menghi, C. (2020). UML consistency rules: a case study with open-source UML models. 8th International Conference on Formal Methods in Software Engineering (FormaliSE 2020).

  • Torres, W., van den Brand, M. G. J., & Serebrenik, A. (2021). A systematic literature review of cross-domain model consistency checking by model management tools. Software and Systems Modeling, 20(3), 897–916.

    Article  Google Scholar 

  • Usman, M., Nadeem, A., Tai-hoon, K., & Eun-suk, C. (2008). A survey of onsistency checking techniques for UML models. Paper presented at the Advanced Software Engineering and Its Applications, Hainan Island, China, December 13–15.

  • Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2000). Experimentation in software engineering: an introduction. Kluwer Academic Publishers.

    Book  MATH  Google Scholar 

  • Wohlin, C., Runeson, P., Hst, M., Ohlsson, M. C., Regnell, B., & Wessln, A. (2012). Experimentation in software engineering. Springer Publishing Company, Incorporated. https://doi.org/10.1007/978-3-642-29044-2

    Article  Google Scholar 

  • Yang, J. (2009). A framework for formalizing UML models with formal language rCOS. Paper presented at the 4th International Conference on Frontier of Computer Science and Technology, Shanghai, China, December 17–19.

Download references

Acknowledgements

This research has been funded by the Natural Sciences and Engineering Research Council of Canada as well as the GEMA project ("Consejería de Educación, Cultura y Deporte de la Dirección General de Universidades, Investigación e Innovación de la JCCM", SBPLY/17/180501/000293), PID2020-112540RB-C42, AETHER-UCLM (A smart data holistic approach for context-aware data analytics), funded by MCIN/AEI/10.13039/501100011033/.

Funding

This research has been funded by the Natural Sciences and Engineering Research Council of Canada as well as the GEMA project (“Consejería de Educación, Cultura y Deporte de la Dirección General de Universidades, Investigación e Innovación de la JCCM,” SBPLY/17/180501/000293), PID2020-112540RB-C42, AETHER-UCLM (A smart data holistic approach for context-aware data analytics), funded by MCIN/AEI/10.13039/501100011033/.

Author information

Authors and Affiliations

  1. A&M University Central Texas, Killeen, TX, USA

    Damiano Torre

  2. University of Castilla-La Mancha, Ciudad Real, Spain

    Marcela Genero

  3. Carleton University, Ottawa, Canada

    Yvan Labiche & Maged Elaasar

Authors
  1. Damiano Torre
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcela Genero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yvan Labiche
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maged Elaasar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Damiano Torre.

Ethics declarations

Conflict of interest

The authors of this paper certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

In order to develop a survey that would adequately gather the information needed to answer the three research questions presented in Sect. 0, we developed a questionnaire consisting of four sections with a total of 21 questions (Q1 to Q21). Figure 14 describes the flow of questions in the questionnaire. The questionnaire had two parts (see Fig. 14): someone who declared to only be an academic followed only the path on the left of Fig. 14. Someone who declared to only be a practitioner followed only the path on the right of Fig. 14. Someone who declared to be both an academic and a practitioner followed both paths: there are 18 such respondents. In this latter case, questions 1–5 and 20–21 are answered only once, whereas questions 6–19 are answered twice but with different flavors: one as academic (left of Fig. 14), one as practitioner (right of Fig. 14). So, there is no overrepresentation of anyone’s opinion during the evaluation of the rules and each respondent’s opinion is only accounted once (questions 20–21). Each one of questions 6–19 asks a question that can be answered differently depending on the context: for instance, one may use different technologies, languages and tools in the classroom from technologies/languages/tools used at work. So, we felt it was important to capture both contexts. When studying and analyzing results we always only analyze results from academia or always only analyze results from industry. So, the opinion of these 18 respondents is always only accounted once.

Fig. 14
figure 14

Series of questions for respondents disclosing industry and/or academic experience

Full size image

The 21 questions can be found in Table 7. We used four different types of answers, as indicated in the table:

  • Single from a list: in this type of question, the questionnaire displays a list of predefined answers which can be selected only one time.

  • Multiple from a list: in this type of question, the questionnaire allows the respondents to select one or more options from a list of predefined answers.

  • Likert scale: in this type of question, the questionnaire allows the respondents to give a rating on a n-pointing agreement scale (Likert scale). The object of the scale is to measure a respondent’s opinion or attitude towards any given subject.

  • Other text: in this type of question, the questionnaire allows respondents to insert free-form text as answer of a specific question.

Table 7 Questionnaire at a glance
Full size table

We used the following Likert scales:

  • 4-point Likert scale for Q9: often, sometimes, rarely, never.

  • 3-point Likert scale for Q10: very important, partially important, not important.

  • 4-point Likert scale for Q12: very familiar, familiar, partially familiar, not familiar.

  • 4-point Likert-scale (plus N/A) for Q13: often, sometimes, rarely, never.

  • 3-point Likert scale for Q21: always, sometimes, never.

Appendix 2

Table 8 lists the set of 52 UML consistency rules, out of the 116 we initially collected, which should always hold in every UML model, as indicated by the majority of the participants of our survey (recall Sect. 0). For each rule, the table provides a unique rule identifier (column #) colored in black for domain-specific rules and left in white for general-purpose rules, a plain-language description (column Rule description), and the UML diagram(s) involved in the rule (column UDI). The rules are presented according to the three phases of the Unified Process considered in this work (recall Sect. 4.4.1: Requirements, Analysis, and Design. The rules that involve two diagrams that belong to two different phases were grouped in mixed phases. For example, rule 98 involves UCD (Requirements) and ID (Analysis), so these rules are grouped as belonging to both Requirements/Analysis phase. Note that rule identifiers are not continuous numbers in this manuscript; We wanted to remain consistent with other publications we have, including our study which lists all 116 rules (Torre et al., 2018).

Table 8 List of the 52 UML consistency rules
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Torre, D., Genero, M., Labiche, Y. et al. How consistency is handled in model-driven software engineering and UML: an expert opinion survey. Software Qual J 31, 1–54 (2023). https://doi.org/10.1007/s11219-022-09585-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-022-09585-2

Keywords

Search

Navigation