Skip to main content
SpringerLink
Log in

Graphical versus textual software measurement modelling: an empirical study

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Model-driven Engineering (MDE) has attained great importance in both the Software Engineering industry and the research community, where it is now widely used to provide a suitable approach with which to improve productivity when developing software artefacts. In this scenario, measurement models (software artefacts) have become a fundamental point in improvement of productivity, where MDE and Software Measurement can reap mutual benefits. MDE principles and techniques can be used in software measurement to build more automatic and generic solutions, and to achieve this, it is fundamental to be able to develop software measurement models. To facilitate this task, a domain-specific language named “Software Measurement Modelling Language” (SMML) has been developed. This paper tackles the question of whether the use of SMML can assist in the definition of software measurement models. An empirical study was conducted, with the aim of verifying whether SMML makes it easier to construct measurement models which are more usable and maintainable as regards textual notation. The results show that models which do not use the language are more difficult—in terms of effort, correctness and efficiency—to understand and modify than those represented with SMML. Additional feedback was also obtained, to verify the suitability of the graphical representation of each symbol (element or relationship) of SMML.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. In http://alarcos.inf-cr.uclm.es/ontologies/smo/—The Web Ontology Language (OWL) is a family of knowledge representation languages for authoring ontologies endorsed by the World Wide Web Consortium.

  2. In http://alarcos.esi.uclm.es/smf/.

  3. In http://alarcos.esi.uclm.es/smf/smtool/.

  4. LabVIEW (short for Laboratory Virtual Instrumentation Engineering Workbench) is a platform and development environment for a visual programming language from National Instruments.

  5. Simulink is an environment for multidomain simulation and Model-Based Design for dynamic and embedded systems. It provides an interactive graphical environment and a customizable set of block libraries that let you design, simulate, implement, and test a variety of time-varying systems, including communications, controls, signal processing, video processing, and image processing.

References

  • Ahmad, R. (1999). Visual languages: A new way of programming. Malaysian Journal of Computer Science, 12(1), 76–81.

    Google Scholar 

  • Apple. (2010a). Automator, from www.macosxautomation.com/automator. Accessed May 2010.

  • Apple. (2010b). appleScript, from http://www.macosxautomation.com/applescript/). Accessed May 2010.

  • Arpaia, P., Buzio, M., Fiscarelli, L., Inglese, V., Commara, G. L., & Walckiers, L. (2009). Measurement-domain specific language for magnetic test specifications at CERN. 2009 IEEE instrumentation and measurement technology conference, CERN/TE 2009-002, 1716–1720.

  • Basili, V. R., Shull, F., & Lanubile, F. (1999). Building knowledge through families of experiments. IEEE Transactions on Software Engineering, 25(4), 456–473.

    Article  Google Scholar 

  • Bézivin, J. (2004). In search of a basic principle for model-driven engineering [Special Issue]. Novatica Journal, 5, 21–24.

    Google Scholar 

  • Bézivin, J., Jouault, F., & Touzet, D. (2005). Principles, standards and tools for model engineering. 10th IEEE international conference on engineering of complex computer systems (ICECCS’2005), 28–29.

  • Cook, S. (2004). Domain-specific modeling and model driven architecture. In D. S. Frankel & J. Parodi (Eds.), The MDA journal: Model driven architecture straight from the masters (Chap. 5, pp. 1–10). Tampa, FL: Meghan-Kiffer Press.

  • Cunniff, N., & Taylor, R. P. (1987). Graphical vs. textual representation: An empirical study of novices’ program comprehension. Empirical studies of programmers: Second workshop, pp. 114–131.

  • Deursen, A. v., Klint, P., & Visser, J. (2000). Domain-specific languages: An annotated bibliography. SIGPLAN Notices, 35(6), 26–36.

    Article  Google Scholar 

  • Eysenck, M. W., & Keane, M. T. (2005). Cognitive psychology: A student’s handbook. Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Feilkas, M. (2006). How to represent models, languages and transformations? Proceedings of the 6th OOPSLA workshop on domain-specific modeling (DSM’06), pp. 204–213.

  • Fenton, N., & Pfleeger, S. L. (1997). Software metrics: A rigorous & practical approach (2nd ed.). Boston, MA: PWS Publishing Company.

    Google Scholar 

  • García, F., Bertoa, M. F., Calero, C., Vallecillo, A., Ruíz, F., Piattini, M., et al. (2006). Towards a consistent terminology for software measurement. Information & Software Technology, 48(8), 631–644.

    Article  Google Scholar 

  • García, F., Ruiz, F., Cruz, J., & Piattini, M. (2003). Integrated measurement for the evaluation and improvement of software processes. Proceedings of the 9th European workshop on software process technology (EWSPT’9), Lecture Notes in Computer Science, 2786, pp. 94–111.

  • García, F., Serrano, M., Cruz-Lemus, J., Ruiz, F., & Piattini, M. (2007). Managing software process measurement: A metamodel-based approach. Information Sciences, 177, 2570–2586.

    Article  Google Scholar 

  • Genero, M., Moody, D., & Piattini, M. (2005). Assessing the capability of internal metrics as early indicators of maintenance effort through experimentation. Journal of Software Maintenance, 17(3), 225–246.

    Article  Google Scholar 

  • Green, T. R. G., Petre, M., & Bellamy, R. K. E. (1991). Comprehensibility of visual and textual programs: A test of superlativism against the match-mismatch conjecture. 4th workshop on empirical studies of programmers, pp. 121–146.

  • Guerra, E., Lara, J. d., & Díaz, P. (2008). Visual specification of measurements and redesigns for domain specific visual languages. Journal of Visual Languages and Computing, 19(3), 399–425.

    Article  Google Scholar 

  • Hull, D., Wolstencroft, K., Stevens, R., Goble, C., Pocock, M., Li, P., & Oin, T. (2006). Taverna: A tool for building and running workflows of services. Nucleic Acids Research, 34(Web Server issue) pp. 729–732.

    Google Scholar 

  • ISO/IEC. (2001). ISO/IEC 9126-1: Software engineering-software product quality—Part 1: Quality model. Geneva, Switzerland, International Organization for Standardization.

  • Jedlitschka, A., & Ciolkowski, M. (2005). Reporting guidelines for controlled experiments in software engineering. ACM/IEEE international symposium on empirical software engineering, pp. 95–195.

  • Jouault, F., Allilaire, F., Bézivin, J., Kurtev, I., & Valduriez, P. (2006). ATL: A QVT-like transformation language. Companion to the 21th annual ACM SIGPLAN conference on object-oriented programming, systems, languages, and applications, OOPSLA 2006, pp. 719–720.

  • Jouault, F., & Bézivin, J. (2006). KM3: A DSL for metamodel specification. Formal methods for open object-based distributed systems, 8th IFIP WG 6.1 international conference, FMOODS 2006, 4037, pp. 171–185.

  • Juristo, N., & Moreno, A. M. (2001). Basics of software engineering experimentation. Boston, MA: Kluwer Academic Publishers.

    MATH  Google Scholar 

  • Karsai, G., Krahn, H., Pinkernell, C., Rumpe, B., Schindler, M., & Völkel, S. (2009). Design guidelines for domain specific languages. The 9th OOPSLA workshop on domain-specific modeling, pp. 7–13.

  • Kelly, S., & Pohjonen, R. (2009). Worst practices for domain-specific modeling. IEEE Software, 26(4), 22–29.

    Article  Google Scholar 

  • Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., Emam, K. E., et al. (2002). Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28(8), 721–734.

    Article  Google Scholar 

  • Kolovos, D. S., Paige, R. F., Kelly, T., & Polack, F. A. C. (2006). Requirements for domain-specific languages. First ECOOP workshop on domain-specific program development (ECOOP’06).

  • Kurtev, I., Bézivin, J., Jouault, F., & Valduriez, P. (2006). Model-based DSL frameworks. OOPSLA Companion 2006, pp. 602–616.

  • Mernik, M., Heering, J., & Sloane, A. M. (2005). When and how to develop domain-specific languages. ACM Computing Surveys (CSUR), 37(4), 316–344.

    Article  Google Scholar 

  • Moher, T. G., Mak, D. C., Blumenthal, B., & Leventahal, L. M. (1993). Comparing the comprehensibility of textual and graphical programs: The case of petri nets. Palo Alto, pp. 137–161.

  • Mora, B., García, F., Ruiz, F., & Piattini, M. (2008b). SMML: Software measurement modeling language. The 8th OOPSLA workshop on domain-specific modeling, pp. 52–59.

  • Mora, B., García, F., Ruiz, F., & Piattini, M. (2009). Model-driven software measurement framework: A case study. The 9th international conference on quality software, QSIC 2009, pp. 239–248.

  • Mora, B., García, F., Ruiz, F., Piattini, M., Boronat, A., Gómez, A., Carsí, J. Á., & Ramos, I. (2008a). Software measurement by using QVT transformation in an MDA context. 10th International conference on enterprise information systems—ICEIS 2008, 1, pp. 117–124.

  • Mora, B., Ruiz, F., Garcia, F., & Piattini, M. (2008c). SMML: Software measurement modeling language, Department of Computer Science. University of Castilla—La Mancha, from http://www.uclm.es/dep/tsi/pdf/SMML_Software_Measurement_Modeling_Language.pdf.

  • Oinn, T., Greenwood, M., Addis, M., Alpdemir, N., Ferris, J., Glover, K., et al. (2006). Taverna: Lessons in creating a workflow environment for the life sciences. Concurrency and Computation: Practice and Experience, 18(10), 1067–1100.

    Article  Google Scholar 

  • OMG. (2003). OCL 2.0—OMG final adopted specification, Object Management Group.

  • OMG. (2005a). Query/view/transformation (QVT) Standard Specification.

  • OMG. (2005b). UML specification: Superstructure version 2.0, Object Management Group, from http://www.omg.org/docs/formal/05-07-04.pdf.

  • Özgür, T. (2007). Comparison of Microsoft DSL tools and eclipse modeling frameworks for domain-specific modeling in the context of the model driven development. School of Engineering. Ronneby, Sweden, Blekinge Institute of Technology: p. 56.

  • Pandey, R. K., & Burnett, M. M. (1993). Is it easier to write matrix manipulation programs visually or textually? An empirical study. IEEE symposium on visual languages, pp. 344–351.

  • Patig, S. (2008). A practical guide to testing the understandability of notations. Fifth Asia-Pacific conference on conceptual modelling (APCCM 2008), pp. 49–55.

  • Pelechano, V., Albert, M., Javier, M., & Carlos, C. (2006). Building tools for model driven development comparing Microsoft DSL tools and eclipse modeling plug-ins. Desarrollo de Software Dirigido por Modelos—DSDM’06.

  • Shneiderman, B., Mayer, R., McKay, D., & Heller, P. (1977). Experimental investigations of the utility of detailed flowcharts in programming. Communications of the ACM, 20(6), 373–381.

    Article  Google Scholar 

  • Shull, F., Carver, J. C., Vegas, S., & Juzgado, N. J. (2008). The role of replications in empirical software engineering. Empirical Software Engineering, 13(2), 211–218.

    Article  Google Scholar 

  • Völter, M. (2009). MD* Best practices. Journal of Object Technology, 8(6), 79–102.

    Article  Google Scholar 

  • Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2000). Experimentation in software engineering: An introduction, International Series in Software Engineering.

  • Yoder, M., & Black, B. (2006). A study of graphical vs. textual programming for teaching DSP. American Society for Engineering Education Annual Conference & Exposition.

  • Zhao, Y., Hategan, M., Clifford, B., Foster, I., Laszewski, G. V., Raicu, I., et al. (2007). Swift: Fast, reliable, loosely coupled parallel computation. 2007 IEEE Congress on Services, pp. 199–206.

Download references

Acknowledgments

This work has been partially supported by the projects: INGENIO (JCCM, PAC08-0154-9262), MEDUSAS (CDTI (MICINN), IDI-20090557), PEGASO/MAGO (MICINN and FEDER, TIN2009-13718-C02-01), and ALTAMIRA (JCCM, Fondo Social Europeo, PII2I09-0106-2463) projects.

Author information

Authors and Affiliations

  1. Indra Software Labs, Information Technology Company, Ciudad Real, Spain

    B. Mora

  2. ALARCOS Research Group, Department of Information Technologies and Systems, University of Castilla-La Mancha, Paseo de la Universidad, 4, 13071, Ciudad Real, Spain

    F. García, F. Ruiz & M. Piattini

Authors
  1. B. Mora
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. F. Ruiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Piattini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to B. Mora.

Appendices

Appendix A: Example of definition of software measurement models

See Figs. 11, 12, 13.

Fig. 11
figure 11

Example of understandability exercise with SMML diagram

Full size image
Fig. 12
figure 12

Example of modifiability exercise with a TEXTUAL notation

Full size image
Fig. 13
figure 13

Example of modifiability exercise with a TEXTUAL notation (cont.)

Full size image

Appendix B: Boxplots

See Figs. 14, 15, 16, 17, 18.

Fig. 14
figure 14

Boxplot diagram of the interaction of UoD × usage of SMML in the experiment for modifiability/understandability valuation

Full size image
Fig. 15
figure 15

Boxplot diagram for the usage of SMML in the experiment for the efficiency

Full size image
Fig. 16
figure 16

Boxplot diagram for the use of SMML in the experiment for the time

Full size image
Fig. 17
figure 17

Boxplot diagram for the use of SMML in the experiment for the correctness

Full size image
Fig. 18
figure 18

Boxplot diagram for the usage of SMML in the experiment for the valuation

Full size image

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mora, B., García, F., Ruiz, F. et al. Graphical versus textual software measurement modelling: an empirical study. Software Qual J 19, 201–233 (2011). https://doi.org/10.1007/s11219-010-9111-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-010-9111-x

Keywords

Search

Navigation