Skip to main content
SpringerLink
Log in

Does the level of detail of UML diagrams affect the maintainability of source code?: a family of experiments

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Although the UML is considered to be the de facto standard notation with which to model software, there is still resistance to model-based development. UML modeling is perceived to be expensive and not necessarily cost-effective. It is therefore important to collect empirical evidence concerning the conditions under which the use of UML makes a practical difference. The focus of this paper is to investigate whether and how the Level of Detail (LoD) of UML diagrams impacts on the performance of maintenance tasks in a model-centric approach. A family of experiments consisting of one controlled experiment and three replications has therefore been carried out with 81 students with different abilities and levels of experience from 3 countries (The Netherlands, Spain, and Italy). The analysis of the results of the experiments indicates that there is no strong statistical evidence as to the influence of different LoDs. The analysis suggests a slight tendency toward better results when using low LoD UML diagrams, especially if used for the modification of the source code, while a high LoD would appear to be helpful in understanding the system. The participants in our study also favored low LoD diagrams because they were perceived as easier to read. Although the participants expressed a preference for low LoD diagrams, no statistically significant conclusions can be drawn from the set of experiments. One important finding attained from this family of experiments was that the participants minimized or avoided the use of UML diagrams, regardless of their LoD. This effect was probably the result of using small software systems from well-known domains as experimental materials.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Abrial JR (1996) The B-Book. Cambridge University Press, New York

    Book  MATH  Google Scholar 

  • Anda B, Hansen K, Gullesen I, Thorsen HK (2006) Experiences from introducing UML-based development in a large safety-critical project. Empir Softw Eng 11:555–581

    Article  Google Scholar 

  • Arisholm E, Briand LC, Hove SE, Labiche Y (2006) The impact of UML documentation on software maintenance: an experimental evaluation. IEEE Trans Softw Eng 32:365–381

    Article  Google Scholar 

  • Basili V, Shull F, Lanubile F (1999) Building knowledge through families of experiments. IEEE Trans Softw Eng 25:456–473

    Article  Google Scholar 

  • Basili V, Weiss D (1984) A methodology for collecting valid software engineering data. IEEE Trans Softw Eng 10:728–738

    Article  Google Scholar 

  • Biostat (2006) Comprehensive Meta-Analysis v2. Biostat, Englewood

    Google Scholar 

  • Briand LC, Labiche Y, Di Penta M, Yan-Bondoc H (2005) An experimental investigation of formality in UML-based development. IEEE Trans Softw Eng 31:833–849

    Article  Google Scholar 

  • Briand LC, Wüst J, Lounis H (2001) Replicated Case Studies for Investigating Quality Factors in Object-Oriented Designs. Empir Softw Eng 6:11–58. doi:10.1023/A:1009815306478

    Article  MATH  Google Scholar 

  • Bruegge B, Dutoit AH (2010) Object-oriented software engineering: using UML, patterns, and Java. Prentice Hall, Boston

    Google Scholar 

  • Cant S, Jeffery D, Henderson-Sellers B (1995) A conceptual model of cognitive complexity of elements of the programming process. Inf Softw Technol 37:351–362. doi:10.1016/0950-5849(95)91491-H

    Article  Google Scholar 

  • Carver J (2010) Towards Reporting Guidelines for Experimental Replications: A Proposal. Proceedings of the 1st International Workshop on Replication in Empirical Software Engineering Research (RESER) [Held during ICSE 2010]

  • Cohen D, Lindvall M, Costa P (2004) An introduction to agile methods. Adv Comput 62:2–67

    Google Scholar 

  • Conover WJ (1998) Practical Nonparametric Statistics, 3rd ed. Wiley

  • Cruz-Lemus JA, Genero M, Caivano D et al (2010) Assessing the influence of stereotypes on the comprehension of UML sequence diagrams: A family of experiments. Inf Softw Technol 53:1391–1403

    Article  Google Scholar 

  • Devore JL, Farnum N (1999) Applied Statistics for Engineers and Scientists. Duxbury Press, NY

    Google Scholar 

  • Dobing B, Parsons J (2006) How UML is used? Commun ACM 49:109–113

    Article  Google Scholar 

  • Dzidek WJ, Arisholm E, Briand LC (2008) A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Trans Softw Eng 34:407–432

    Article  Google Scholar 

  • Erickson J, Siau K (2007) Theoretical and practical complexity of modeling methods. Commun ACM 50:46–51

    Article  Google Scholar 

  • Ericksson HE, Penker M, Lyons B, Fado D (2004) UML 2 Toolkit. Wiley

  • Fernández-Sáez AM, Chaudron MRV, Genero M, Ramos I (2013a) Are forward designed or reverse-engineered UML diagrams more helpful for code maintenance?: a controlled experiment. Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering. ACM, New York, pp 60–71

    Google Scholar 

  • Fernández-Sáez AM, Genero M, Chaudron MRV (2013b) Empirical studies concerning the maintenance of UML diagrams and their use in the maintenance of code: A systematic mapping study. Inf Softw Technol 55:1119–1142

    Article  Google Scholar 

  • Fernández-Sáez AM, Genero M, Chaudron MRV et al (2014) Are Forward Designed or Reverse-Engineered UML diagrams more helpful for code maintenance?: A family of experiments. Inf Softw Technol. doi:10.1016/j.infsof.2014.05.014

    Google Scholar 

  • Fernández-Sáez AM, Genero M, Chaudron MRV (2012) Does the Level of Detail of UML Models Affect the Maintainability of Source Code? Proceedings of the Experiences and Empirical Studies in Software Modelling Workshop (EESSMod’11) at MODELS 2011. LNCS 7167, Wellington, New Zealand, pp 133–147

  • Fjeldstad RK, Hamlen WT (1979) Application Program Maintenance Study: Report to Our Respondents. Proceedings of GUIDE 48

  • Garousi G, Garousi V, Moussavi M, et al. (2013) Evaluating Usage and Quality of Technical Software Documentation: An Empirical Study. Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering (EASE’2013). ACM, pp 24–35

  • Glass R (2002) Facts and fallacies of software engineering. Addison-Wesley

  • Glässer U, Gotzhein R, Prinz A (2003) The formal semantics of SDL-2000: Status and perspectives. Comput Netw 42:343–358. doi:10.1016/S1389-1286(03)00247-0

    Article  MATH  Google Scholar 

  • Gravino C, Tortora G, Scanniello G (2010) An empirical investigation on the relation between analysis models and source code comprehension. Proceedings of the 2010 ACM Symposium on Applied Computing (SAC’2010). ACM, pp 2365–2366

  • Grossman M, Aronson JE, McCarthy RV (2005) Does UML make the grade? Insights from the software development community. Inf Softw Technol 47:383–397

    Article  Google Scholar 

  • Hannay JE, Dybå T, Arisholm E, Sjøberg DIK (2009) The effectiveness of pair programming: A meta-analysis. Inf Softw Technol 51:1110–1122

    Article  Google Scholar 

  • Hedges LV, Olkin I (1985) Statistical Methods for Meta-Analysis. Academia Press, New York

    MATH  Google Scholar 

  • Höst M, Regnell B, Wholin C (2000) Using students as subjects - a comparative study of students and professionals in lead-time impact assessment. Proceedings of the 4th Conference on Empirical Assessment and Evaluation in Software Engineering. pp 201–214

  • ISO/IEC (2014) ISO/IEC 25001: Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - Planning and management. International Organization for Standarization

  • Jedlitschka A, Ciolkowoski M, Pfahl D (2008) Reporting experiments in software engineering. Guide to Advanced Empirical Software Engineering

  • Juristo N, Moreno A (2001) Basics of software engineering experimentation. Kluwer Academic Publishers, Boston

    Book  MATH  Google Scholar 

  • Juristo N, Vegas S, Solari M et al (2013) A process for managing interaction between experimenters to get useful similar replications. Inf Softw Technol 55:215–225

    Article  Google Scholar 

  • Kampenes V, Dybå T, Hannay JE, Sjoberg DIK (2007) A Systematic Review of Effect Size in Software Engineering Experiments. Inf Softw Technol 49:1073–1086

    Article  Google Scholar 

  • Karahasanovic A, Thomas R (2007) Difficulties experienced by students in maintaining object-oriented Systems: an empirical study. Proceedings of the Australasian Computing Education Conference (ACE’2007). pp 81–87

  • Kirk RE (1995) Experimental design. procedures for the behavioural sciences. Brooks/Cole Publishing Company, Belmont

    Google Scholar 

  • Kitchenham BA, Pfleeger S, Hoaglin DC et al (2002) Preliminary Guidelines for Empirical Research in Software Engineering. IEEE Trans Softw Eng 28:721–734

    Article  Google Scholar 

  • Lange CFJ, Chaudron MRV, Muskens J (2006) In practice: UML software architecture and design description. IEEE Softw 23:40–46

    Article  Google Scholar 

  • Lauesen S (2002) Software Requirements: Styles and Techniques. Addison-Wesley, UK

    Google Scholar 

  • Lientz BP, Swanson EB (1980) Software Maintenance Management. Addison -Wesley, Massachusetts

    Google Scholar 

  • Lindsay RM, Ehrenberg A (1993) The design of replicated studies. Am Stat 47:217–228

    Google Scholar 

  • Nugroho A (2009) Level of detail in UML models and its impact on model comprehension: A controlled experiment. Inf Softw Technol 51:1670–1685

    Article  Google Scholar 

  • Nugroho A, Chaudron MRV (2009) Evaluating the impact of UML modeling on software quality: An industrial case study. LNCS 5795:181–195

    Google Scholar 

  • Nugroho A, Chaudron MRV (2008) A survey into the rigor of UML use and its perceived impact on quality and productivity. Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement (ESEM 2008). ACM, New York, pp 90–99

    Google Scholar 

  • OMG (2010) The Unified Modeling Language. Documents associated with UML version 2.3. http://www.omg.org/spec/UML/2.3.

  • Oppenheim AN (2000) Questionnaire Design, Interviewing and Attitude Measurement, 0002-New ed. Bloomsbury Academic, United Kingdom

    Google Scholar 

  • Pippenger N (1978) Complexity Theory. Scientific American 238:

  • Poels G, Dedene G (2000) Measures for Assessing Dynamic Complexity Aspects of Object-oriented Conceptual Schemes. Proceedings of the 19th International Conference on Conceptual Modeling. Springer, Berlin, pp 499–512

    Google Scholar 

  • Pressman RS (2005) Software engineering: a practitioners approach, seventh. McGraw Hill, New York

    Google Scholar 

  • Roehm T, Tiarks R, Koschke R, Maalej W (2012) How Do Professional Developers Comprehend Software? Proceedings of the 34th International Conference on Software Engineering. IEEE Press, Piscataway, pp 255–265

    Google Scholar 

  • Scanniello G, Gravino C, Genero M, et al. (2013) On the Impact of UML Analysis Models on Source Code Comprehensibility and Modifiability. ACM Transactions On Software Engineering And Methodology (In press) 26

  • Scanniello G, Gravino C, Tortora G (2012) Does the Combined use of Class and Sequence Diagrams Improve the Source Code Comprehension? Results from a Controlled Experiment. Proceedings of the Experiences and Empirical Studies in Software Modelling Workshop (EESSMoD´2012)

  • Scanniello G, Gravino C, Tortora G (2010) Investigating the Role of UML in the Software Modeling and Maintenance - A Preliminary Industrial Survey. Proceedings of the 12th International Conference on Enterprise Information Systems (ICEIS’2010). Funchal, Madeira, Portugal, pp 141–148

  • Sheskin D (2007) Handbook of Parametric and Nonparametric Statistical Procedures, 4th edn. Chapman and Hall, New York

    MATH  Google Scholar 

  • Siau K (1999) Information Modeling and Method Engineering: A Psychological Perspective. J Database Manag 10:44–50

    Article  Google Scholar 

  • Singer J, Vinson NG (2002) Ethical Issues in Empirical Studies of Software Engineering. IEEE Trans Softw Eng 28:1171–1180. doi:10.1109/TSE.2002.1158289

    Article  Google Scholar 

  • Sjøberg DIK, Hannay JE, Hansen O et al (2005) A survey of controlled experiments in software engineering. IEEE Trans Softw Eng 31:733–753

    Article  Google Scholar 

  • Spivey JM (1989) The Z Notation: A Reference Manual. Prentice-Hall, New York

    MATH  Google Scholar 

  • SPSS (2003) SPSS 12.0, syntax reference guide. SPSS Inc, Chicago

    Google Scholar 

  • Tryggeseth E (1997) Report from an Experiment: Impact of Documentation on Maintenance. J Empir Softw Eng 2:201–207

    Article  Google Scholar 

  • Vegas S, Juristo N, Moreno A, et al. (2006) Analysis of the influence of communication between researchers on experiment replication. Proceedings of the ACM/IEEE international symposium on Empirical software engineering (ISESE’2006). pp 28–37

  • Verelst J (2004) The influence of the level of abstraction on the evolvability of conceptual models of information systems. Proceedings of the International Symposium on Empirical Software Engineering (ISESE’04). pp 17–26

  • Wieringa RJ (2003) Chapter 8 - Entity-Relationship Diagrams. In: Wieringa RJ (ed) Design Methods for Reactive Systems. Morgan Kaufmann, San Francisco, pp 77–88

    Chapter  Google Scholar 

  • Wohlin C, Runeson P, Host M et al (1999) Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, Boston

    Google Scholar 

Download references

Acknowledgments

This work has been funded by the following projects: GEODAS-BC (Ministerio de Economía y Competitividad and Fondo Europeo de Desarrollo Regional FEDER, TIN2012-37493-C03-01) and IMPACTUM (Consejería de Educación, Ciencia y Cultura de la Junta de Comunidades de Castilla La Mancha, y Fondo Europeo de Desarrollo Regional FEDER, PEII11-0330-4414).

Author information

Authors and Affiliations

  1. ALARCOS Research Group, Department of Technologies and Information Systems, University of Castilla-La Mancha, Paseo de la Universidad 4, 13071, Ciudad Real, Spain

    Ana M. Fernández-Sáez & Marcela Genero

  2. Department of Informatics, University of Bari, Via E. Orabona 4, 70126, Bari, Italy

    Danilo Caivano

  3. SER&Practices s.r.l, Spin Off Company of the University of Bari, Via E. Orabona 4, 70126, Bari, Italy

    Danilo Caivano

  4. Joint Computer Science and Engineering Department of Chalmers, University of Technology and University of Gothenburg, SE-412 96, Gö̈teborg, Sweden

    Michel R. V. Chaudron

  5. Leiden Institute of Advanced Computer Sciences, Leiden University, NielsBohrweg 1, 2333, CA, Leiden, The Netherlands

    Ana M. Fernández-Sáez & Michel R. V. Chaudron

Authors
  1. Ana M. Fernández-Sáez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcela Genero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Danilo Caivano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michel R. V. Chaudron
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana M. Fernández-Sáez.

Additional information

Communicated by: Richard Paige

Appendix

Appendix

Fig. 16
figure 16

Example of a low LoD class diagram

Full size image
Fig. 17
figure 17

Example of a high LoD class diagram

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fernández-Sáez, A.M., Genero, M., Caivano, D. et al. Does the level of detail of UML diagrams affect the maintainability of source code?: a family of experiments. Empir Software Eng 21, 212–259 (2016). https://doi.org/10.1007/s10664-014-9354-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-014-9354-4

Keywords

Search

Navigation