Skip to main content
SpringerLink
Log in

Assessing the understandability of UML statechart diagrams with composite states—A family of empirical studies

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

The main goal of this work is to present a family of empirical studies that we have carried out to investigate whether the use of composite states may improve the understandability of UML statechart diagrams derived from class diagrams. Our hypotheses derive from conventional wisdom, which says that hierarchical modeling mechanisms are helpful in mastering the complexity of a software system. In our research, we have carried out three empirical studies, consisting of five experiments in total. The studies differed somewhat as regards the size of the UML statechart models, though their size and the complexity of the models were chosen so that they could be analyzed by the subjects within a limited time period. The studies also differed with respect to the type of subjects (students vs. professionals), the familiarity of the subjects with the domains of the diagrams, and other factors. To integrate the results obtained from each of the five experiments, we performed a meta-analysis study which allowed us to take into account the differences between studies and to obtain the overall effect that the use of composite states has on the understandability of UML statechart diagrams throughout all the experiments. The results obtained are not completely conclusive. They cast doubts on the usefulness of composite states for a better understanding and memorizing of UML statechart diagrams. Composite states seem only to be helpful for acquiring knowledge from the diagrams. At any rate, it should be noted that these results are affected by the previous experience of the subjects on modeling, as well as by the size and complexity of the UML statechart diagrams we used, so care should be taken when generalizing our results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Even though understandability is not a maintainability sub-characteristic by the ISO 9126 standard (ISO/IEC, 2001), software quality research considers understandability to be a main factor influencing maintainability (Briand et al. 2001; Fenton and Pfleeger 1997; Harrison et al. 2000).

References

  • Agarwal R, De P, Sinha AP (1999) Comprehending Object and Process Models: An Empirical Study. IEEE Trans Softw Eng 25(4):541–556 doi:10.1109/32.799953

    Article  Google Scholar 

  • Arisholm E, Sjøberg DIK (2004) Evaluating the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software. IEEE Trans Softw Eng 30(8):521–534 doi:10.1109/TSE.2004.43

    Article  Google Scholar 

  • Atkinson C, Kühne T (2003) Model Driven Development: a Metamodeling Foundation. IEEE Trans Softw Eng 20:36–41

    Google Scholar 

  • Basili V, Shull F, Lanubile F (1999) Building Knowledge through Families of Experiments. IEEE Trans Softw Eng 25:456–473 doi:10.1109/32.799939

    Article  Google Scholar 

  • Biostat, Inc. (2006) Meta-Analysis v2. http://www.meta-analysis.com

  • Bodart F, Patel A, Sim M, Weber R (2001) Should Optimal Properties Be Used in Conceptual Modelling? A Theory and Three Empirical Tests. Inf Syst Res 12(4):384–405 doi:10.1287/isre.12.4.384.9702

    Article  Google Scholar 

  • Bolloju N, Leung FSK (2006) Assisting Novice Analysts in Developing Quality Conceptual Models with UML. Commun ACM 49(7):108–112 doi:10.1145/1139922.1139926

    Article  Google Scholar 

  • Briand L, Bunse C, Daly J (2001) A Controlled Experiment for Evaluating Quality Guidelines on the Maintainability of Object-Oriented Designs. IEEE Trans Softw Eng 27(6):513–530 doi:10.1109/32.926174

    Article  Google Scholar 

  • Carter J, Jaccheri L, Morasca S, Shull F (2003) Issues in Using Students in Empirical Studies in Software Engineering Education. Proc. 9th International Software Metrics Symposium (METRICS’ 03), Sydney, Australia, pp 239–249

    Google Scholar 

  • Craig SD, Gholson B, Driscoll DM (2002) Animated Pedagogical Agents in Multimedia Educational Environments: Effects of Agent Properties, Picture Features, and Redundancy. J Educ Psychol 94(2):428–434 doi:10.1037/0022-0663.94.2.428

    Article  Google Scholar 

  • Cruz-Lemus JA, Genero M, Manso ME, Piattini M (2005a) Evaluating the Effect of Composite States on the Understandability of UML Statechart Diagrams. Proc. 8th International Conference on Model-Driven Engineering, Languages and Systems (MoDELS 2005), Montego Bay, Jamaica, pp 113–125

    Google Scholar 

  • Cruz-Lemus JA, Genero M, Piattini M (2005b) Metrics for UML Statechart Diagrams. In Metrics for Software Conceptual Models. Imperial College Press, UK

    Google Scholar 

  • Cruz-Lemus JA, Genero M, Piattini M, Toval Álvarez JA (2005c) An Empirical Study of the Nesting Level of Composite States within UML Statechart Diagrams. Proc. First International Workshop on Best Practices of UML (BP-UML 2005)-ER 2005 Workshops, Klagenfurt (Austria), pp 12–22

    Google Scholar 

  • Denger C, Ciolkowski M (2003) High Quality Statecharts through Tailored, Perspective-Based Inspections. Proc. 29th EUROMICRO Conference “New Waves in System Architecture”, Belek, Turkey, pp 316–325

    Google Scholar 

  • Dybå T, Arisholm E, Sjøberg DIK, Hannay JE, Shull F (2007) Are Two Heads Better than One? On the Effectiveness of Pair Programming. IEEE Softw 24(6):10–13 doi:10.1109/MS.2007.158

    Article  Google Scholar 

  • Fenton N, Pfleeger S (1997) Software Metrics: A Rigurous and Practical Approach. International Thompson Publishing Inc, London

    Google Scholar 

  • Firesmith D, Henderson-Sellers B, Graham I (1998) OPEN modeling language (OML) reference manual. Cambridge University Press, New York, USA

    Google Scholar 

  • Gemino A, Wand Y (2003) Evaluating Modeling Techniques based on Models of Learning. Commun ACM 46(10):79–84 doi:10.1145/944217.944243

    Article  Google Scholar 

  • Gemino A, Wand Y (2005) Complexity and Clarity in Conceptual Modeling: Comparison of Mandatory and Optional Properties. Data Knowl Eng 55:301–326 doi:10.1016/j.datak.2004.12.009

    Article  Google Scholar 

  • Genero M, Manso ME, Visaggio A, Piattini M, Canfora G (2007) Building Measure-based Prediction Models for UML Class Diagram Maintainability. Empir Softw Eng 12(5):527–549 doi:10.1007/s10664-007-9038-4

    Article  Google Scholar 

  • Genero M, Cruz-Lemus JA, Caivano D, Abrahao S, Insfran E, Carsí JA (2008) Assessing the Influence of Stereotypes on the Comprehension of UML Sequence Diagrams: A Controlled Experiment. Lecture Notes in Computer Science (5301): 11th ACM/IEEE MODELS Conference, 280–294, October

  • Glass GV, McGaw B, Smith ML (1981) Meta-Analysis in Social Research. Sage Publications.

  • Glezer C, Last M, Nachmany E, Shoval P (2005) Quality and Comprehension of UML Interaction Diagrams—An Experimental Comparison. Inf Softw Technol 47:675–692 doi:10.1016/j.infsof.2005.01.003

    Article  Google Scholar 

  • Harrison R, Counsell S, Nithi R (2000) Experimental Assessment of the Effect of Inheritance on the Maintainability of Object-Oriented Systems. J Syst Softw 52:173–179 doi:10.1016/S0164-1212(99)00144-2

    Article  Google Scholar 

  • Hayes W (1999) Research in Software Engineering: a Case for Meta-Analysis. Proc. 6th IEEE International Symposium on Software Metrics (METRICS’ 99), Boca Raton, USA, pp 143–151

    Google Scholar 

  • Hedges LV, Olkin I (1985) Statistical Methods for Meta-Analysis. Academia Press.

  • Höst M, Regnell B, Wohlin C (2000) Using Students as Subjects—a Comparative Study of Students & Profesionals in Lead-Time Impact Assessment. Proc. 4th Conference on Empirical Assessment & Evaluation in Software Engineering (EASE 2000), Keele, UK, pp 201–214

    Google Scholar 

  • Hu Z, Shatz SM (2006) Explicit Modeling of Semantics Associated with Composite States in UML Statecharts. Autom Softw Eng 13(4):423–467 doi:10.1007/s10515-006-0272-6

    Article  Google Scholar 

  • ISO/IEC (2001) Software Product Evaluation-Quality Characteristics and Guidelines for their Use. ISO/IEC Standard 9126.

  • Juristo N, Moreno A (2001) Basics of Software Engineering Experimentation. Kluwer Academic Publishers.

  • Kampenes V, Dybå T, Hannay JE, Sjoberg DIK (2007) A Systematic Review of Effect Size in Software Engineering Experiments. Inf Softw Technol 49(11–12):1073–1086 doi:10.1016/j.infsof.2007.02.015

    Article  Google Scholar 

  • Khatri V, Vessey I, Ramesh PCV, Park SJ (2006) Understanding Conceptual Schemas: Exploring the Role of Application and IS Domain Knowledge. Inf Syst Res 17(1):81–99 doi:10.1287/isre.1060.0081

    Article  Google Scholar 

  • Kirk RE (1995) Experimental Design: Procedures for the Behavioral Sciences. Brooks/Cole Publishing Company.

  • Kitchenham B, Pfleeger S, Pickard L, Jones P, Hoaglin D, El-Emam K, Rosenberg J (2002) Preliminary Guidelines for Empirical Research in Software Engineering. IEEE Trans Softw Eng 28(8):721–734 doi:10.1109/TSE.2002.1027796

    Article  Google Scholar 

  • Laitenberger O, El-Emam K, Harbich T (1999) An Internally Replicated Quasy-Experimental Comparison of Checklist and Perspective-based Reading of Code Documents. 006.99/e. IESE.

  • Lipsey M, Wilson D (2001) Practical Meta-Analysis, Sage.

  • Miller J (2000) Applying Meta-Analytical Procedures to Software Engineering Experiments. J Syst Softw 54:29–39 doi:10.1016/S0164-1212(00)00024-8

    Article  Google Scholar 

  • Pickard LM (2004) Combining Empirical Results in Software Engineering, Univer-sity of Keele, T-R V1.

  • Mayer RE (1989) Models for Understanding. Rev Educ Res 59(1):43–64

    Google Scholar 

  • Mayer RE, Anderson RB (1991) Animations need Narrations: An Experimental Test of the Dual-Coding Hypothesis. J Educ Psychol 83(4):484–490 doi:10.1037/0022-0663.83.4.484

    Article  Google Scholar 

  • Mayer RE (2001) Multimedia Learning. Cambridge University Press.

  • Miller J, McDonald F (1998) Statistical Analysis of Two Experimental Studies. EFoCS-31-98. University of Strathclyde.

  • OMG (2003) UML 2.0-2nd Revised Submission. document ptc/03-01-07. Object Management Group.

  • Otero MC, Dolado JJ (2004) Evaluation of the Comprehension of the Dynamic Modeling in UML. Inf Softw Technol 46(1):35–53 doi:10.1016/S0950-5849(03)00108-3

    Article  Google Scholar 

  • Otero MC, Dolado JJ (2005) An Empirical Comparison of the Dynamic Modeling in OML and UML. J Syst Softw 77(2):91–102 doi:10.1016/j.jss.2004.11.022

    Article  Google Scholar 

  • Pigoski T (1997) Practical Software Maintenance. New York, USA: Wiley Computer Publishing

    Google Scholar 

  • Porter A, Johnson M (1997) Assessing Software Review Measurement: Necessary and Sufficient Properties for Software Measures. Inf Softw Technol 42(1):35–46

    Google Scholar 

  • Purchase HC, Colpoys L, McGill M, Carrington D, Britton C (2001) UML Class Diagram Syntax: an Empirical Study of Comprehension. Proc. Australian Symposium on Information Visualisation, Sydney, Australia, pp 113–120

    Google Scholar 

  • Purchase HC, Colpoys L, McGill M, Carrington D (2002) UML Collaboration Diagram Syntax: an Empirical Study of Comprehension. Proc. 1st International Workshop on Visualizing Software for Understanding and Analysis (VISSOFT’02), Paris, France, pp 13–22

    Google Scholar 

  • Ricca F, Di Penta M, Torchiano M, Tonella P, Ceccato M (2007) The Role of Experience and Ability in Comprehension Tasks supported by UML Stereotypes. Proc. 29th International Conference on Software Engineering (ICSE, 07), Minneapolis, USA, pp 375–384

    Google Scholar 

  • Rosenthal R (1986) Meta-Analytic Procedures for Social Research. Sage Publications.

  • Rosenthal R (1994) Parametric measures of effect size. In The Handbook of Research Synthesis. Russell Sage Foundation, New York

    Google Scholar 

  • Selic B (2003) The Pragmatics of Model-Driven Development. IEEE Softw 20(5):19–25 doi:10.1109/MS.2003.1231146

    Article  Google Scholar 

  • Sjoberg DIK, Hannay JE, Hansen O, Kampenes V, Karahasanovic A, Liborg NK, Rekdal AC (2005) A Survey of Controlled Experiments in Software Engineering. IEEE Trans Softw Eng 31(9):733–753 doi:10.1109/TSE.2005.97

    Article  Google Scholar 

  • SPSS (2003) SPSS 12.0, Syntax Reference Guide. SPSS Inc, Chicago, USA

    Google Scholar 

  • Staron M, Kuzniarz L, Wohlin C (2006) Empirical Assessment of Using Stereotypes to Improve Comprehension of UML Models: a Set of Experiments. J Syst Softw 79:727–742 doi:10.1016/j.jss.2005.09.014

    Article  Google Scholar 

  • Sutton JA, Abrams RK, Jones RD, Sheldon AT, Song F (2001) Methods for Meta-Analysis in Medical Research. John-Wiley & Sons.

  • Tabbers HK (2004) Multimedia Instructions and Cognitive Load Theory: Effects of Modality and Cueing. Br J Educ Psychol 74(1):71–81 doi:10.1348/000709904322848824

    Article  Google Scholar 

  • Thomas D (2004) MDA: Revenge of the Modelers or UML Utopia. IEEE Softw 21(3):15–17 doi:10.1109/MS.2004.1293067

    Article  Google Scholar 

  • Tichy WF (2000) Hints for Reviewing Empirical Work in Software Engineering. Empir Softw Eng 5:309–312 doi:10.1023/A:1009844119158

    Article  MathSciNet  Google Scholar 

  • Verelst J (2004) The Influence of the Level of Abstraction on the Evolvability of Conceptual Models of Information Systems. Proc. 3rd International Symposium on Empirical Software Engineering (ISESE 2004), Redondo Beach, USA, pp 17–26

    Google Scholar 

  • Webb K (2006) Xholon Digital Watch Project. http://www.primordion.com/Xholon/samples/watch.htm

  • Winer BJ, Brown DR, Michels KM (1991) Statistical Principles in Experimental Design. McGraw-Hill.

  • Wohlin C, Runeson P, Hast M, Ohlsson MC, Regnell B, Wesslen A (2000) Experimentation in Software Engineering: an Introduction. Kluwer Academic Publisher.

  • Wolf FM (1986) Meta-Analysis: Quantitative Methods for Research Synthesis. Sage Publications.

  • Woodfield SN, Dunsmore HE, Shen VY (1981) The Effect of Modularization and Comments on Program Comprehension. Proc. 5th International Conference on Software Engineering (ICSE 1981), San Diego, USA, pp 215–223

    Google Scholar 

  • Xie S, Kraemer E, Stirewalt REK (2007) Empirical Evaluation of a UML Sequence Diagram with Adornments to Support Understanding of Thread Interactions. Proc. 15th IEEE International Conference on Program Comprehension (ICPC’07), Banff, Canada, pp 123–134

    Google Scholar 

  • Yusuf S, Kagdi H, Maletic JI (2007) Assessing the Comprehension of UML Class Diagrams via Eye Tracking. Proc. 15th IEEE International Conference on Program Comprehension (ICPC’07), Banff, Canada, pp 113–122

    Google Scholar 

Download references

Acknowledgements

This research is part of the IDONEO project (PAC08-0160-6141) financed by “Consejería de Ciencia y Tecnología de la Junta de Comunidades de Castilla-La Mancha” and the ESFINGE project supported by the “Ministerio de Educación y Ciencia (Spain)” (TIN2006-15175-C05-05). The research presented in this paper has been partially funded by the IST project “QualiPSo,” sponsored by the EU in the 6th FP (IST-034763), the FIRB project “ARTDECO,” sponsored by the Italian Ministry of Education and University, and the project “La qualità nello sviluppo software,” sponsored by the Università degli Studi dell’Insubria.

The authors would like to thank:

● Professors Ambrosio Toval (University of Murcia) and Cristina Cachero (University of Alicante) and their students for having cooperated in the performance of the experiments.

● Our PhD students at the Universidad de Castilla-La Mancha (Spain) and our undergraduate students at the Università degli Studi dell’Insubria (Italy) for their unselfish help in the second experiment and its replication.

● The staff in INDRA (formerly Soluziona)—Ciudad Real (Spain) for their time and understanding during the preparation for and performance of the third experiment.

● The reviewers involved in the development process of this paper; all your valuable comments have helped us improve its quality.

Author information

Authors and Affiliations

  1. Department of Technologies and Information Systems, University of Castilla-La Mancha, Paseo de la Universidad, 4, Ciudad Real, 13071, Spain

    José A. Cruz-Lemus, Marcela Genero & Mario Piattini

  2. Departament of Computer Science, University of Valladolid, Campus Miguel Delibes, E.T.I.C., Valladolid, 47011, Spain

    M. Esperanza Manso

  3. Dipartimento di Scienze della Cultura, Politiche e dell’Informazione, Università degli Studi dell’Insubria, Como, Italy

    Sandro Morasca

Authors
  1. José A. Cruz-Lemus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcela Genero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Esperanza Manso
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sandro Morasca
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mario Piattini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José A. Cruz-Lemus.

Appendices

Appendix A. Experimental Material

All the experimental material concerning the family of experiments is available on-line at http://alarcos.esi.uclm.es/CSExperiments/

By way of example, we include the diagrams and tests used in the third experiment (E3).

PHASE 1: The Xholon Watch

The Xholon Digital Watch (© 2005, 2006 Ken Webb) sample application simulates the internal structure and behavior of a fairly generic digital watch.

It has four buttons which can be labeled S1, S2, S3 and S4. It displays the time or the date. By pressing the four buttons in various combinations, the current second, minute, hour, month, day and date can be updated, and various other internal functions can be managed. Figure 7

Fig. 7
figure 7

The Xholon Watch

Full size image

The main functionalities of the buttons are outlined in the following table:

Button

Functionality

S1

Time / Date

S2

Set Alarm on / off

S3 normal

Chronometer

S3 long

Date/Time/Alarm Update

S4

Light

On the following page, you will find the state machine that modeled how the watch works. Please look at this and try to understand it. Take as much time as you need.

Once you are ready, start answering the questions in test #1. You must answer the questions in the same order as they are given and note down the exact time at which you finish answering the last question.

To complete this phase, you can look at the diagram as many times as you need.

Once you have finished each of the phases of the experiment, please remain seated and wait for further instructions from the experiment supervisor.

PHASE 1: The Xholon Watch. Test #1

Please answer the following questions in the same order they are given. Put a “T” in the blank box with if you think that the sentence is true and an “F” if you think that it is false.

Do not forget to note down the time at which you finish answering the last question.

Thank you very much. You may begin.

1. If we are in the state TIME and the button S1 is pressed twice we reach the state TIME again.

2. There is only one possible combination of buttons to set the alarm off.

3. The chronometer may be running while the date is displayed on the watch.

4. When updating the date and time we can increase and decrease the values by pressing several buttons.

5. If button S3 is pressed for 2 s while the date is being displayed, we get to the alarm update mode.

6. The order in which the alarm, date and time are updated is always the same.

7. The diagram models how the light of the watch works.

8. While updating the alarm, date and time, the real time can be displayed at any moment by pressing one button.

9. Whenever a button is pressed, there is a transition between states.

10. There is a specific button combination to change the day of the week we are in.

TEST FINISHED IN:________minutes_________seconds

PHASE 1: The Xholon Watch. Test #2

In this phase, you will find a paragraph that describes the main features of the watch. You must complete each of the blanks in the text with a suitable word or group of words that make the text complete and meaningful.

Do not forget to write down the time after you have finished exercise.

Thank you very much. You may begin.

The Xholon DW application simulates the internal structure and the (1) of a conventional digital watch.

It displays the time or the date. By pressing the four buttons in various combinations, the current second, minute, hour, month, day and date can be updated, and various other internal functions can be managed.

It has four buttons which can be labeled S1, S2, S3 and S4. By pressing the four buttons in various combinations, the watch displays the time or the (2).

The current second, minute, hour, month, day and date can be updated, and various other internal functions can be managed, such as (3), for example.

The watch displays the date by pressing and releasing the button (4). It also allows us to change the alarm status, and there can be a beep on every (5) or when the (6) is set on. Both options may also be set on or off together.

The (7) has its typical functionalities and can be left running while the time is displayed. To change from one mode to another, button (8) must be pressed.

To update the date and time, we must press the button (9), and change the value for the alarm, time and date by pressing the button (10) to increase the displayed value in 1 unit and the button (11) to pass from one element to another.

TEST FINISHED IN: minutes seconds

PHASE 1: The Xholon Watch. Test #3

In this test, you must perform a series of tasks that will be described next. You have some blank sheets for this purpose and you can use them and bring them back at the end of this test.

When performing the required tasks, you can comment on as many aspects of your solution as you want.

Please do not forget to note down the time at which you finish carrying out the last of the tasks.

Thank you very much. You may begin.

  1. 1.

    Build a state machine that models the possibility of the buttons S1 and S2 being pressed together.

  2. 2.

    Taking as a starting point the time 00hs00mis of January 1st 2000, please indicate the minimum sequence of buttons to be pressed to update the watch to 03hs07mins of June 13th 2005.

  3. 3.

    What would happen if the following sequence of buttons is pressed while the watch is displaying the time?

    • ◆S1

    • ◆S1

    • ◆S2

    • ◆S1

    • ◆S1

    • ◆S2

  4. 4.

    Please model the behavior of button S4, which turns the watch light on and off, by adding to the original diagram as many states, transitions, events, guard conditions and activities as you consider necessary.

  5. 5.

    Please indicate the minimum sequence of buttons to be pressed to update the alarm to 08hs00mins and activate it, but not the chime.

  6. 6.

    What would happen if the following sequence of buttons is pressed while the watch is displaying the time?

    • ◆S3

    • ◆S1

    • ◆S2

    • ◆S1

    • ◆S2

TEST FINISHED IN:______minutes________seconds

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cruz-Lemus, J.A., Genero, M., Manso, M.E. et al. Assessing the understandability of UML statechart diagrams with composite states—A family of empirical studies. Empir Software Eng 14, 685–719 (2009). https://doi.org/10.1007/s10664-009-9106-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-009-9106-z

Keywords

Search

Navigation