Skip to main content
SpringerLink
Log in

Evaluating the Quality of Process Models: Empirical Testing of a Quality Framework

  • Conference paper
  • First Online:
Book cover Conceptual Modeling — ER 2002 (ER 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2503))

Included in the following conference series:

Abstract

This paper conducts an empirical analysis of a conceptual model quality framework for evaluating the quality of process models. 194 participants were trained in the concepts of the quality framework, and then used it to evaluate models represented in a workflow modelling language. A randomised, double-blind design was used, and the results evaluated using a combination of quantitative and qualitative techniques. An analysis was also conducted of the framework’s likelihood of adoption in practice, which is an issue rarely addressed in IS design research. The study provides strong support for the validity of the framework and suggests that it is likely to be adopted in practice, but raises questions about its reliability. The research findings provide clear direction for further research to improve the framework.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alexander, C. (1968): Notes On The Synthesis Of Form, Boston: Harvard University Press.

    Google Scholar 

  2. Babbie, E.R. (1998): The Practice of Social Research, Belmont, California: Wadsworth Publishing.

    Google Scholar 

  3. Batini, C, S. Ceri, and S.B. Navathe (1992): Conceptual Database Design: An Entity Relationship Approach, Redwood City, California: Benjamin Cummings.

    MATH  Google Scholar 

  4. Batra, D., J.A. Hoffer, and R.P. Bostrom (1990): Comparing Representations with Relational and EER Models, Communications of the ACM, 33(2), February: p. 126–139.

    Google Scholar 

  5. Batra, D. and J. Davis (1992): Conceptual Data Modelling In Database Design: Similarities And Differences Between Expert And Novice Designers, International Journal Of Man-Machine Studies, 37: p. 83–101.

    Article  Google Scholar 

  6. Bloom, B. (1984): Taxonomy of Educational Objectives, New York: Longman.

    Google Scholar 

  7. Bock, D.B. and T. Ryan (1993): Accuracy in Modelling with Extended Entity Relationship and Object Oriented Data Model, Journal Of Database Management, 4(4): p. 30–39.

    Google Scholar 

  8. Bodart, F., A. Patel, M. Sim, and R. Weber (2001): Should the Optional Property Construct be used in Conceptual Modelling: A Theory and Three Empirical Tests, Information Systems Research, 12(4).

    Google Scholar 

  9. Bubenko, J.A. (1986): Information Systems Methodologies-A Research View, In T.W. Olle, H.G. Sol, and A.A. Verrijn-Stuart (Eds.), Information Systems Design Methodologies: Improving The Practice: North-Holland.

    Google Scholar 

  10. Chaiyasut, P. and G.G. Shanks (1994): Conceptual Data Modelling Process: A Study Of Novice And Expert Data Modellers, In T. Halpin and R. Meersman (Eds.), Proceedings Of The 1st International Conference On Object Role Modelling, Magnetic Island, Queensland, Australia, July.

    Google Scholar 

  11. Chau, P.Y.K. (1996): An Empirical Assessment of a Modified Technology Acceptance Model, Journal of Management Information Systems, 13(2).

    Google Scholar 

  12. Cooper, D.R. and P.S. Schindler (1998): Business Research Methods (6th edition), 6th ed, Singapore: McGraw-Hill International.

    Google Scholar 

  13. Curtis, B. (1986): By The Way, Did Anyone Study Any Real Programmers?, In Empirical Studies of Programmers, E. Soloway and S. Iyengar (Eds.), Ablex: Norward, N.J.

    Google Scholar 

  14. Davis, F.D., R.P. Bagozzi, and P.R. Warshaw (1989): User Acceptance of Computer Technology: A Comparison of Two Theoretical Models, Management Science, 35(8): p. 982–1003.

    Article  Google Scholar 

  15. Fitzgerald, G. (1991): Validating New Information Systems Techniques: A Retrospective Analysis, In H.E. Nissen, H.K. Klein, and R. Hirschheim (Eds.), Information Systems Research: Contemporary Approaches And Emergent Traditions: North-Holland.

    Google Scholar 

  16. Galliers, R.D. (1994): Relevance and Rigour in Information Systems Research: Some Personal Reflections on Issues Facing the Information Systems Research Community, In Proceedings of the IFIP TC8 Conference on Business Process Reengineering: Information Systems and Challenges, Gold Coast, Australia.

    Google Scholar 

  17. Hardgrave, B.C. and N.P. Dalal (1995): Comparing Object Oriented and Extended Entity Relationship Models, Journal Of Database Management, 6(3).

    Google Scholar 

  18. Hu, P.J. and P.Y.K. Chau (1999): Examining the Technology Acceptance Model Using Physician Acceptance of Telemedicine Technology, Journal of Management Information Systems, 16(2), Fall: p. 91–113.

    Google Scholar 

  19. Ivari, J. (1986): Dimensions Of Information Systems Design: A Framework For A Long Range Research Program, Information Systems Journal, (June).

    Google Scholar 

  20. Kaplan, B. and D. Duchon (1988): Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study, MIS Quarterly, December.

    Google Scholar 

  21. Keen, P.G.W. (1991): Relevance and Rigour in Information Systems Research: Improving Quality, Confidence, Cohesion and Impact, In Information Systems Research: Contemporary Approaches and Emergent Traditions, H.-E. Nissen, H.K. Klein, and R. Hirschheim (Eds.), Elsevier Science Publishers (North Holland).

    Google Scholar 

  22. Kesh, S. (1995): Evaluating The Quality Of Entity Relationship Models, Information And Software Technology, 37(12).

    Google Scholar 

  23. Krogstie, J. (1998): Integrating the Understanding of Quality in Requirements Specification and Conceptual Modelling, Software Engineering Notes, 28(1).

    Google Scholar 

  24. Lauesen, S. and O. Vinter (2000): Preventing Requirement Defects, In Proceedings of the Sixth International Workshop on Requirements Engineering: Foundation for Software Quality (REFSQ’2000), Stockholm, Sweden, June 5-6 2000.

    Google Scholar 

  25. Lee, H. and B.G. Choi (1998): A Comparative Study of Conceptual Data Modelling Techniques, Journal Of Database Management, 9(2), Spring: p. 26–35.

    Google Scholar 

  26. Levitin, A. and T. Redman (1994): Quality Dimensions of a Conceptual View, Information Processing and Management, 31(1).

    Google Scholar 

  27. Lindland, O.I., G. Sindre, and A. Solvberg (1994): Understanding Quality In Conceptual Modelling, IEEE Software, 11(2), March: p. 42–49.

    Google Scholar 

  28. Maiden, N. and A. Sutcliffe (1992): Analysing the Novice Analyst: Cognitive Models in Software Engineering, International Journal Of Man Machine Studies, 367: p. 719–740.

    Google Scholar 

  29. Martin, J. (1989): Information Engineering, Englewood Cliffs, N.J.: Prentice Hall, 3 v.

    Google Scholar 

  30. Moody, D.L. and G.G. Shanks (1994): What Makes A Good Data Model? Evaluating the Quality of Entity Relationship Models, In P. Loucopolous (Ed.) Proceedings of the Thirteenth International Conference on the Entity Relationship Approach, Manchester, England, December 14–17.

    Google Scholar 

  31. Moody, D.L. (1998): Metrics for Evaluating the Quality of Entity Relationship Models, In T.W. Ling, S. Ram, and MX. Lee (Eds.), Proceedings of the Seventeenth International Conference on Conceptual Modelling (ER’ 98), Singapore: Elsevier Lecture Notes in Computer Science, November 16–19.

    Google Scholar 

  32. Moody, D.L. and G.G. Shanks (1998): Evaluating and Improving the Quality of Entity Relationship Models: An Action Research Programme, Australian Computer Journal, November.

    Google Scholar 

  33. Moody, D.L. and G.G. Shanks (1998): What Makes A Good Data Model? A Framework For Evaluating And Improving The Quality Of Entity Relationship Models, Australian Computer Journal, August.

    Google Scholar 

  34. Moody, D.L., G.G. Shanks, and P. Darke (1998): Evaluating and Improving the Quality of Entity Relationship Models: Experiences in Research and Practice, In T.W. Ling, S. Ram, and MX. Lee (Eds.), Proceedings of the Seventeenth International Conference on Conceptual Modelling (ER’ 98), Singapore: Elsevier Lecture Notes in Computer Science, November 16–19.

    Google Scholar 

  35. Moody, D.L. (2000): Building Links Between IS Research and Professional Practice: Improving the Relevance and Impact of IS Research, In R.A. Weber and B. Glasson (Eds.), International Conference on Information Systems (ICIS’00), Brisbane, Australia, December 11–13.

    Google Scholar 

  36. Moody, D.L. (2000): Strategies for Improving the Quality of Entity Relationship Models, In Information Resource Management Association (IRMA) Conference, Anchorage, Alaska: Idea Group Publishing, May 21–24.

    Google Scholar 

  37. Moody, D.L. (2001): Dealing with Complexity: A Practical Method for Representing Large Entity Relationship Models (PhD Thesis), Melbourne, Australia: Department Of Information Systems, University of Melbourne.

    Google Scholar 

  38. Morris, C.W. (1970): Foundations of the Theory of Signs, Chicago: Chicago University Press.

    Google Scholar 

  39. Neuman, WX. (2000): Social Research Methods-Qualitative and Quantitative Approaches (4th edition), Needham Heights, MA: Allyn and Bacon.

    Google Scholar 

  40. Nordbotten, J.C. and M.E. Crosby (1999): The Effect of Graphic Style on Data Model Interpretation, Information Systems Journal, 9.

    Google Scholar 

  41. Nunally, J. (1978): Psychometric Theory (2nd edition), New York: McGraw Hill.

    Google Scholar 

  42. Olle, T.W., H.G. Sol, and A.A. Verrijn-Stuart, eds. Information Systems Design Methodologies: A Comparative Review. 1982, North-Holland: Amsterdam.

    Google Scholar 

  43. Olle, T.W., H.G. Sol, and C.J. Tully, eds. Information Systems Design Methodologies: A Feature Analysis. 1983, North-Holland: Amsterdam.

    Google Scholar 

  44. Olle, T.W., H.G. Sol, and A.A. Verrijn-Stuart, eds. Information Systems Design Methodologies: Improving the Practice. 1986, North-Holland: Amsterdam.

    Google Scholar 

  45. Rosemann, M., W. Sedera, and D. Sedera (2001): Testing a Framework for the Quality of Process Models: A Case Study, In Fifth Pacific Asia Conference on Informaiton Systems (PACIS’2001), Seoul, Korea

    Google Scholar 

  46. Schuette, R. and T. Rotthowe (1998): The Guidelines Of Modelling: An Approach To Enhance The Quality In Information Models, In Proceedings Of The Seventeenth International Conference On Conceptual Modelling (ER’ 98), Singapore: Elsevier Lecture Notes in Computer Science, November 16–19.

    Google Scholar 

  47. Shanks, G.G., G.C. Simsion, and M. Rembach (1993): The Role Of Experience In Conceptual Data Modelling, In Fourth Australian Information Systems Conference, Brisbane, Australia

    Google Scholar 

  48. Shoval, P. and M. Even-Chaime (1987): Database Schema Design: An Experimental Comparison Between Normalisation and Information Analysis, Database, 18(3), Spring: p. 30–39.

    Article  Google Scholar 

  49. Shoval, P. and S. Shiran (1997): Entity-Relationship and Object-Oriented Data Modeling: An Experimental Comparison of Design Quality, Data & Knowledge Engineering, 21: p. 297–315.

    Article  MATH  Google Scholar 

  50. Standish Group (1995): The CHAOS Report into Project Failure, The Standish Group International Inc. Available on-line at http://www.s1andishgroup.com/visitor/chaos.htm.

  51. Standish Group (1996): Unfinished Voyages, The Standish Group International Inc. available on-line at http://www.standishgroup.com/visitor/voyages.htm.

  52. van Vliet, H. (2000): Software Engineering: Principles and Practice (2nd edition): John Wiley & Sons.

    Google Scholar 

  53. von Halle, B. (1991): Data: Asset or Liability?, Database Programming and Design, 4(7), July.

    Google Scholar 

  54. Weber, R.A. (1996): Are Attributes Entities? A Study Of Database Designers’ Memory Structures, Information Systems Research, June.

    Google Scholar 

  55. Weber, R.A. (1997): Ontological Foundations Of Information Systems, Melbourne, Australia: Coopers And Lybrand Accounting Research Methodology Monograph No. 4, Coopers And Lybrand.

    Google Scholar 

  56. Westrup, C. (1993): Information Systems Methodologies in Use, Journal of Information Technology, 8.

    Google Scholar 

  57. Witt, G.C. and G.C. Simsion (2000): Data Modeling Essentials: Analysis, Design, and Innovation, The Coriolis Group.

    Google Scholar 

  58. Wynekoop, J.L. and NX. Russo (1997): Studying Systems Development Methodologies: An Examination Of Research Methods, Information Systems Journal, 7(1), January.

    Google Scholar 

  59. Zultner, R.E. (1992): The Deming Way: Total Quality Management for Software, In Proceedings of Total Quality Management for Software Conference, Washington, DC, April.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, Norwegian University of Science and Technology (NTNU), Trondheim, Norway, N-7491

    Daniel L. Moody, Guttorm Sindre, Terje Brasethvik & Arne Sølvberg

  2. School of Business Systems, Monash University, Melbourne, Australia, 3800

    Daniel L. Moody

Authors
  1. Daniel L. Moody
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guttorm Sindre
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Terje Brasethvik
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arne Sølvberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Database Laboratory, School of Computer and Communication Sciences, Ecole Polytechnique Fédérale Lausanne (EPFL), 1015, Lausanne, Switzerland

    Stefano Spaccapietra

  2. Owen Graduate School of Management, Vanderbilt University, Nashville, TN, 37203, USA

    Salvatore T. March

  3. Graduate School of Informatics, Department of Social Informatics, Kyoto University, Yoshida Sakyo, Kyoto, 606-8501, Japan

    Yahiko Kambayashi

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Moody, D.L., Sindre, G., Brasethvik, T., Sølvberg, A. (2002). Evaluating the Quality of Process Models: Empirical Testing of a Quality Framework. In: Spaccapietra, S., March, S.T., Kambayashi, Y. (eds) Conceptual Modeling — ER 2002. ER 2002. Lecture Notes in Computer Science, vol 2503. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45816-6_36

Download citation

  • DOI: https://doi.org/10.1007/3-540-45816-6_36

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-44277-6

  • Online ISBN: 978-3-540-45816-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation