Skip to main content
SpringerLink
Log in

Evaluating the effort of composing design models: a controlled experiment

  • Special Section Paper
  • Published:
Software & Systems Modeling Aims and scope Submit manuscript

Abstract

Model composition plays a key role in many tasks in model-centric software development, e.g., evolving UML diagrams to add new features or reconciling models developed in parallel by different software development teams. However, based on our experience in previous empirical studies, one of the main impairments for the widespread adoption of composition techniques is the lack of empirical knowledge about their effects on developers’ effort. This problem applies to both existing categories of model composition techniques, i.e., specification-based (e.g., Epsilon) and heuristic-based techniques (e.g., IBM RSA). This paper, therefore, reports on a controlled experiment that investigates the effort of (1) applying both categories of model composition techniques and (2) detecting and resolving inconsistencies in the output composed models. We evaluate the techniques in 144 evolution scenarios, where 2,304 compositions of elements of UML class diagrams were produced. The main results suggest that (1) the employed heuristic-based techniques require less effort to produce the intended model than the chosen specification-based technique, (2) there is no significant difference in the correctness of the output composed models generated by these techniques, and (3) the use of manual heuristics for model composition outperforms their automated counterparts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. France, R., Rumpe, B.: Model-driven development of complex software: a research roadmap. In: Future of Software Engineering at ICSE’07, pp. 37–54 (2007)

  2. Mens, T.: A state-of-the-art survey on software merging. IEEE Trans. Softw. Eng. 28(5), 449–462 (2002)

    Article  Google Scholar 

  3. Clarke, S.: Composition of object-oriented software design models. Ph.D. thesis, Dublin City University (2001)

  4. Epsilon.: http://www.eclipse.org/gmt/epsilon/ (2013)

  5. IBM Rational Software Architect. http://www.ibm.com/developerworks/rational/products/rsa/ (2013)

  6. Farias, K.: Empirical evaluation of effort on composing design models. Ph.D. thesis, PUC-Rio, Rio de Janeiro, Brazil (2012)

  7. Farias, K.: Empirical evaluation of effort on composing design models. In: 32nd ACM/IEEE International Conference on Software Engineering, Doctoral Symposium, vol. 2, pp. 405–408, Cape Town, South Africa (2010)

  8. Clarke, S., Baniassad, E.: Aspect-Oriented Analysis and Design: The Theme Approach. Addison-Wesley, Upper Saddle River (2005)

    Google Scholar 

  9. Farias, K., Garcia, A., Whittle, J., Chavez, C., Lucena, C.: Evaluating the effort of composing design models: a controlled experiment, In: Proceedings of the 15th International Conference on Model-Driven Engineering Languages and Systems (MODELS’12), Innsbruck, Austria, vol. 7590, pp. 676–691 (2012)

  10. Farias, K., Garcia, A., Whittle, J.: Assessing the impact of aspects on model composition effort. In: 9th International Conference on Aspect-Oriented Software Development (AOSD’12), Saint Malo, France, pp. 73–84 (2010)

  11. Clarke, S., Walker, R.: Composition patterns: an approach to designing reusable aspects. In: 23rd International Conference on Software Engineering (ICSE’01), Toronto, ON, pp. 5–14 (2001)

  12. Ricca, F., Penta, M., Torchiano, M., Tonella, P., Ceccato, M.: How developers’ experience and ability influence web application comprehension tasks supported by UML stereotypes: a series of four experiments. IEEE Trans. Softw. Eng. 96(1), 96–118 (2010)

    Article  Google Scholar 

  13. Wohlin, C., et al.: Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, Norwell (2000)

    Book  Google Scholar 

  14. Cook, T., Campbell, D., Day, A.: Quasi-experimentation: Design and Analysis Issues for Field Settings. Houghton Mifflin, Boston (1979)

    Google Scholar 

  15. Kitchenham, B., Al-Kilidar, H., Babar, M., Berry, M., Cox, K., Keung, J., Kurniawati, F., Staples, M., Zhang, H., Zhu, L.: Evaluating guidelines for reporting empirical software engineering studies. Empir. Softw. Eng. 13(1), 97–112 (2008)

    Article  Google Scholar 

  16. Kitchenham, B.: Empirical paradigm: the role of experiments. In: Empirical Software Engineering Issues, pp. 25–32 (2006)

  17. Sjøberg, D., Anda, B., Arisholm, E., Dybå, T., Jørgensen, M., Karahasanovic, A., Koren, E., Vokác, M.: Conducting realistic experiments in software engineering. In: 1st International Symposium on Empirical Software Engineering, pp. 17–26 (2002)

  18. Basili, V., Caldiera, G., Rombach, H.: The Goal Question Metric Paradigm. Encyclopedia of Software Engineering. Wiley, New York (1994)

    Google Scholar 

  19. Devore, J., et al.: Applied Statistics for Engineers and Scientists. Duxbury, North Scituate (1999)

    Google Scholar 

  20. Campbell, D., Russo, M.: Social Experimentation. SAGE Classics, Beverly Hills (1998)

    Google Scholar 

  21. Research method knowledge base: improving conclusion validity. http://www.socialresearchmethods.net/kb/concimp.php (2011)

  22. Jørgensen, M.: Practical guidelines for expert-judgment-based software effort estimation. IEEE Software, pp. 57–63 (2005)

  23. Brewer, M.: Research design and issues of validity. In: Reis, H., Judd, C. (eds.) Handbook of Research Methods in Social and Personality Psychology. Cambridge University Press, Cambridge (2000)

  24. Shadish, W., Cook, T., Campbell, D.: Experimental and Quasi-experimental Designs for Generalized Causal Inference. Houghton Mifflin, Boston (2002)

    Google Scholar 

  25. Mitchell, M., Jolley, J.: Research Design Explained, 4th edn. Harcourt, New York (2001)

    Google Scholar 

  26. Whittle, J., Jayaraman, P.: Synthesizing hierarchical state machines from expressive scenario descriptions. ACM Trans. Softw. Eng. Methodol. 19(3), 1–45 (2010)

  27. Thaker, S., Batory, D., Kitchin, D., Cook, W.: Safe composition of product lines. In: 6th Generative Programming: Concepts and Experiences, Salzburg, Austria, pp. 95–104 (2007)

  28. Klein, J., Hélouët, L., Jézéquel, J., Semantic-based weaving of scenarios. In: 5th Aspect-Oriented Software Development (AOSD’06), Bonn, Germany, pp. 27–38 (2006)

  29. Asklund, U.: Identifying inconsistencies during structural merge. In: Proceedings of Nordic Workshop Programming Environment Research, pp. 86–96 (1994)

  30. Farias, K., Garcia, A., Lucena, C.: Effects of stability on model composition effort: an exploratory study. J. Softw. Syst. Model. (SoSym) 12(1), 1–22 (2013)

  31. Norris, N., Letkeman, K.: Governing and managing enterprise models: part 1. Introduction and concepts. IBM Developer Works, www.ibm.com/developerworks/rational/library/09/0113_letkeman-norris (2011)

  32. Perry, D., Siya, P., Votta, L.: Parallel changes in large scale software development: an observational case study. In: International Conference on Software Engineering (ICSE’98), pp. 251–260 (1998)

  33. Lange, C., Chaudron, M.: Effects of defects in UML models: an experimental investigation. In: International Conference on Software Engineering (ICSE’06), China, pp. 401–410 (2006)

  34. Petre, M.: UML in practice. In: 35th International Conference on Software Engineering (ICSE 2013), San Francisco, CA, pp. 18–26 (2013)

  35. Briand, L., Labiche, Y., Di Penta, M., BondocL, H.: An experimental investigation of formality in UML-based development. IEEE Trans. Softw. Eng. 31(10), 833–849 (2005)

    Article  Google Scholar 

  36. Unified Modeling Language: Infrastructure, Object Management Group (2010)

Download references

Author information

Authors and Affiliations

  1. Interdisciplinary Postgraduate Program in Applied Computing (PIPCA), University of Vale do Rio dos Sinos (Unisinos), São Leopoldo, RS, Brazil

    Kleinner Farias

  2. OPUS Research Group, LES, Informatics Department, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, RJ, Brazil

    Alessandro Garcia & Carlos Lucena

  3. School of Computing and Communications, Lancaster University, City of Lancaster, UK

    Jon Whittle

  4. Department of Computer Science, Federal University of Bahia, Salvador, BA, Brazil

    Christina von Flach Garcia Chavez

Authors
  1. Kleinner Farias
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandro Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jon Whittle
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christina von Flach Garcia Chavez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carlos Lucena
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kleinner Farias.

Additional information

Communicated by Dr. Jürgen Kazmeier.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Farias, K., Garcia, A., Whittle, J. et al. Evaluating the effort of composing design models: a controlled experiment. Softw Syst Model 14, 1349–1365 (2015). https://doi.org/10.1007/s10270-014-0408-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-014-0408-2

Keywords

Search

Navigation