skip to main content
10.1145/2460999.2461008acmotherconferencesArticle/Chapter ViewAbstractPublication PageseaseConference Proceedingsconference-collections
research-article

Are forward designed or reverse-engineered UML diagrams more helpful for code maintenance?: a controlled experiment

Published: 14 April 2013 Publication History

Abstract

Context: UML has been the de facto standard notation for modeling object-oriented software systems since its appearance in 1997. UML diagrams are important for maintainers of a system, especially when the software was developed by a different team. These diagrams of the system are not always available, however, and are commonly recovered using Reverse Engineering (RE) techniques. When obtained through RE, UML diagrams have a high level of detail as compared to those developed in the forward design activity. Method: In this paper we report on a comparison of the attitude and performance of maintainers when using these two kinds of diagrams during the maintenance of source code. Our findings were obtained by carrying out a controlled experiment with 40 students of a Master's degree in Computer Science. Results: The results show a preference for forward design diagrams but do not display significant differences in task performance. The post-experiment survey results have led us to conclude that the subjects did not consider RE diagrams helpful; they found them difficult to understand, particularly the sequence diagrams. In the case of forward design diagrams, subjects considered sequence diagrams as useful, but they did not really employ them. Conclusions: Based on our findings, as regards performance of maintainers, there are no objective results which favor the use of one of these types of diagram in particular, i.e., UML diagrams which come from forwards design, on the one hand, and diagrams obtained from RE, on the other. Subjective opinions do, however, lead us to recommend the use of diagrams created during design. Nevertheless, we realize that the results should be considered as preliminary ones; further replications of this experiment are planned, using students and professionals, the aim being to obtain more conclusive results.

References

[1]
Arisholm, E., Briand, L. C., Hove, S. E. and Labiche, Y. The Impact of UML Documentation on Software Maintenance: An Experimental Evaluation. IEEE Transaction on Software Engineering, 32 (6). 365--381.
[2]
Basili, V., Shull, F. and Lanubile, F. Building knowledge through families of experiments. IEEE Transactions on Software Engineering, 25 (4). 456--473.
[3]
Basili, V. and Weiss, D. A methodology for collecting valid software engineering data. IEEE Transactions on Software Engineering, 10 (6). 728--738.
[4]
Devore, J. L. and Farnum, N. Applied Statistics for Engineers and Scientists. Duxbury, 1999.
[5]
Dobing, B. and Parsons, J. How UML is used? Communications of the ACM, 49 (5). 109--114.
[6]
Dzidek, W. J., Arisholm, E. and Briand, L. C. A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Transactions on Software Engineering, 34 (3). 407--432.
[7]
Erickson, J. and Siau, K. Theoretical and practical complexity of modeling methods. Communications of the ACM, 50 (8). 46--51.
[8]
Fernández-Sáez, A. M., Genero, M. and Chaudron, M. R. V. Does the level of detail of UML models affect the maintainability of source code? Proceedings of the 2011th international conference on Models in Software Engineering (MODELS'2011), Experiences and Empirical Studies in Software Modelling Workshop (EESSMod), Wellington, New Zealand, 2011, 134--148.
[9]
Genero, M., Fernández-Sáez, A. M., Nelson, H. J., Poels, G. and Piattini, M. A systematic literature review on the quality of UML models. Journal of Database Management, 22 (3). 46--70.
[10]
Glass, R. Facts and fallacies of software engineering. Addison-Wesley, 2002.
[11]
Gravino, C., Tortora, G. and Scanniello, G. An empirical investigation on the relation between analysis models and source code comprehension ACM Symposium on Applied Computing (SAC'2010), ACM, Sierre, Switzerland, 2010, 2365--2366.
[12]
Grossman, M., Aronson, J. and McCarthy, R. Does UML make the grade? Insights from the software development community. Information and Software Technology, 47 (6). 383--397.
[13]
Heijstek, W. and Chaudron, M. R. V. Empirical Investigations of Model Size, Complexity and Effort in Large Scale, Distributed Model Driven Development Processes - A Case Study 35th Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2009), Patras, Greece, 2009.
[14]
Höst, M., Regnell, B. and Wohlin, C. Using students as subjects - a comparative study of students and professionals in lead-time impact assessment. Empirical Software Engineering, 5 (3). 201--214.
[15]
Hutchinson, J., Whittle, J., Rouncefield, M. and Kristoffersen, S. Empirical Assessment of MDE in Industry Proceedings of the 33rd International Conference on Software Engineering (ICSE'11), ACM, New York, NY, USA, 2011, 471--480
[16]
ISO/IEC ISO/IEC 14764-1999: Software Engineering Maintenance.
[17]
Jedlitschka, A., Ciolkowoski, M. and Pfahl, D. Reporting experiments in software engineering. in Shull, F., Singer, J. and Sjøberg, D. I. K. eds. Guide to Advanced Empirical Software Engineering Springer Verlag, 2008.
[18]
Juristo, N. and Moreno, A. Basics of software engineering experimentation. Kluwer Academic Publishers, 2001.
[19]
Karahasanovic, A. and Thomas, R. Difficulties experienced by students in maintaining object-oriented Systems: an empirical study Australasian Computing Education Conference (ACE'2007) 2007, 81--87.
[20]
Kirk, R. E. Experimental design. procedures for the behavioural sciences. Brooks/Cole Publishing Company, 1995.
[21]
Lange, C. F. J., Chaudron, M. R. V. and Muskens, J. In practice: UML software architecture and design description. IEEE Software, 23 (2). 40--46.
[22]
Nugroho, A. Level of detail in UML models and its impact on model comprehension: A controlled experiment. Information and Software Technology, 51 (12). 1670--1685.
[23]
Nugroho, A. and Chaudron, M. R. V. Evaluating the impact of UML modeling on software quality: An industrial case study 12th International Conference on Model Driven Engineering Languages and Systems (MODELS'09), Springer, Denver, CO, USA, 2009.
[24]
Nugroho, A. and Chaudron, M. R. V. A survey into the rigor of UML use and its perceived impact on quality and productivity Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement (ESEM 2008), ACM, New York, NY, USA, 2008, 90--99.
[25]
OMG. The Unified Modeling Language. Documents associated with UML version 2.3: http://www.omg.org/spec/UML/2.3/, Object Management Group, 2010.
[26]
Oppenheim, A. N. Questionnaire design, interviewing and attitude measurement. Pinter Publishers, 1992.
[27]
Pressman, R. S. Software engineering: a practitioners approach. McGraw Hill, 2005.
[28]
Scanniello, G., Gravino, C. and Tortora, G. Investigating the role of UML in the software modeling and maintenance - a preliminary industrial survey 12th International Conference on Enterprise Information Systems, Funchal, Madeira, Portugal, 2010, 141--148.
[29]
Sheskin, D. Handbook of Parametric and Nonparametric Statistical Procedures. Chapman and Hall, 2007.
[30]
Shull, F., Mendonça, M. G., Basili, V. R., Carver, J., Maldonaldo, J. C., Fabbri, S., Travassos, G. H. and Ferreira, M. C. Knowledge-sharing issues in experimental software engineering. Empirical Software Engineering, 9 (1--2). 111--137.
[31]
Sjøberg, D. I. K., Hannay, J. E., Hansen, O., Kampenes, V. B., Karahasanovic, A., Liborg, N. and Rekdal, A. C. A survey of controlled experiments in software engineering. IEEE Transaction on Software Engineering, 31 (9). 733--753.
[32]
SPSS. SPSS 12.0, syntax reference guide. SPSS Inc., Chicago, USA, 2003.
[33]
Swanson, E. B. The dimensions of maintenance Proceedings of the 2nd international conference on Software engineering (ICSE 1976), IEEE Computer Society Press, San Francisco, California, United States, 1976, 492--497.
[34]
Van Vliet, H. Software engineering: principles and practices Wiley, 2008.
[35]
Vegas, S., Juristo, N., Moreno, A., Solari, M. and Letelier, P. Analysis of the influence of communication between researchers on experiment replication Proceedings of the ACM/IEEE international symposium on Empirical software engineering (ISESE'2006), 2006, 28--37.
[36]
Verelst, J. The influence of abstraction on the evolvability of conceptual models of information systems International Symposium on Empirical Software Engineering (ISESE'04), 2004, 17--26.
[37]
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B. and Wesslén, A. Experimentation in software engineering: an Introduction. Kluwer Academic Publishers, Norwell, MA, USA, 2000.

Cited By

View all
  • (2019)Software Models for Source Code Maintainability: A Systematic Literature Review2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA.2019.00047(252-259)Online publication date: Aug-2019
  • (2018)Program comprehension through reverse‐engineered sequence diagramsJournal of Software: Evolution and Process10.1002/smr.196530:11Online publication date: 14-Nov-2018
  • (2017)What is the Further Evidence about UML? - A Systematic Literature Review2017 24th Asia-Pacific Software Engineering Conference Workshops (APSECW)10.1109/APSECW.2017.28(106-113)Online publication date: Dec-2017
  • Show More Cited By

Index Terms

  1. Are forward designed or reverse-engineered UML diagrams more helpful for code maintenance?: a controlled experiment

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    EASE '13: Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering
    April 2013
    268 pages
    ISBN:9781450318488
    DOI:10.1145/2460999
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    • Centro de Informatica - UFPE: Centro de Informatica - UFPE
    • SBC: Brazilian Computer Society
    • CNPq: Conselho Nacional de Desenvolvimento Cientifico e Tecn
    • CAPES: Brazilian Higher Education Funding Council

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 14 April 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. UML diagrams
    2. controlled experiment
    3. reverse engineering
    4. software maintenance
    5. survey

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    EASE '13
    Sponsor:
    • Centro de Informatica - UFPE
    • SBC
    • CNPq
    • CAPES

    Acceptance Rates

    EASE '13 Paper Acceptance Rate 31 of 94 submissions, 33%;
    Overall Acceptance Rate 71 of 232 submissions, 31%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)14
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2019)Software Models for Source Code Maintainability: A Systematic Literature Review2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA.2019.00047(252-259)Online publication date: Aug-2019
    • (2018)Program comprehension through reverse‐engineered sequence diagramsJournal of Software: Evolution and Process10.1002/smr.196530:11Online publication date: 14-Nov-2018
    • (2017)What is the Further Evidence about UML? - A Systematic Literature Review2017 24th Asia-Pacific Software Engineering Conference Workshops (APSECW)10.1109/APSECW.2017.28(106-113)Online publication date: Dec-2017
    • (2016)Surveys in Software EngineeringProceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2961111.2962632(1-6)Online publication date: 8-Sep-2016
    • (2016)Helping Program Comprehension of Large Software Systems by Identifying Their Most Important ClassesEvaluation of Novel Approaches to Software Engineering10.1007/978-3-319-30243-0_7(122-140)Online publication date: 2016
    • (2015)Are Forward Designed or Reverse-Engineered UML diagrams more helpful for code maintenance?: A family of experimentsInformation and Software Technology10.1016/j.infsof.2014.05.01457(644-663)Online publication date: Jan-2015
    • (2014)Sampling improvement in software engineering surveysProceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2652524.2652566(1-4)Online publication date: 18-Sep-2014
    • (2014)Condensing class diagrams by analyzing design and network metrics using optimistic classificationProceedings of the 22nd International Conference on Program Comprehension10.1145/2597008.2597157(110-121)Online publication date: 2-Jun-2014
    • (2014)Condensing reverse engineered class diagrams through class name based abstraction2014 4th World Congress on Information and Communication Technologies (WICT 2014)10.1109/WICT.2014.7077321(158-163)Online publication date: Dec-2014
    • (2014)Interactive Scalable Abstraction of Reverse Engineered UML Class DiagramsProceedings of the 2014 21st Asia-Pacific Software Engineering Conference - Volume 0110.1109/APSEC.2014.34(159-166)Online publication date: 1-Dec-2014
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media