Nora Taibouni
ABSTRACT
An information system (IS) measures toolbox is proposed for evaluating implemented IS. The toolbox supports the evaluator in his action from the definition stage to the results one. The software is based on the content, context, process (CCP) framework with adaptations and the IS systemic view examination. An IS evaluation models and measures library is integrated to the toolbox in order to give material to evaluators about previous studies and their results when addressing the main evaluation questions what is being evaluated, with what measures, why evaluation is being done, who is interested by the evaluation and how to conduct evaluation. The toolbox makes possible the cumulative tradition to be carried forward within the discipline.
- Bailey E.J. and Pearson S.W. 1983. Development of a Tool for Measuring and Analyzing Computer User Satisfaction. Management Science, vol.29, n. 5, p. 530--545. https://www.jstor.org/stable/2631354Google Scholar
- Baroudi J.J. and Orlikowski W.J., 1988. A Short Form Measure of User Information Satisfaction: a Psychometric Evaluation and Notes on Use. Journal of Management Information Systems, vol.4, n. 4, p. 44--59. https://www.jstor.org/stable/41432887Google ScholarDigital Library
- Brynjolfsson E. and Hitt L. M. 1996. Paradox Lost? Firmlevel Evidence on the Returns to Information Systems Spending. Management Science, vol.42, p. 541--558. https://pdfs.semanticscholar.org/a89f/4018cc37c1473646dc8cafa106adb9bbe463.pdfGoogle ScholarDigital Library
- Brynjolfsson E. and Hitt L. M., 2000. Beyond Computation: Information Technology, Organizational Transformation and Business Performance. The Journal of Economics Perspectives, vol.14, n. 4, p. 23--48. https://www.jstor.org/stable/2647074Google ScholarCross Ref
- Davis F.D., 1989 Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, vol.13, n. 3, p. 319--340. https://www.jstor.org/stable/249008Google ScholarDigital Library
- Davis G.B. 2000 Information Systems Conceptual Foundations: Looking Backward and Forward. In: Baskerville R., Stage J., DeGross J.I. (eds) Organizational and Social Perspectives on Information Technology. IFIP --- The International Federation for Information Processing, vol 41. Springer, Boston, MA perspectives on information technology. Springer US, 61--82. DOI: https://doi. org/10.1007/978-0-387-35505-4_5Google Scholar
- Davis, G.B. and Olson, M.H., 1985, Management information systems: Conceptual foundations, structure and development, (2nd edn), McGraw-Hill, New York, N.Y.Google Scholar
- Davis F.D., Venkatesh V., 1996. A critical Assessment of Potential Measurement Biases in the Technology Acceptance Model: Three Experiments. International Journal of Human-Computer Studies, vol.45, n. 1, p. 19--45. https://doi.org/10.1006/ijhc.1996.0040Google ScholarDigital Library
- Delone W.H. McLean E.R., 1992.Information Systems Success: The Quest for the Dependent Variable. Information Systems Research, vol.3, n. 1, p. 60--95. https://doi.org/10.1287/isre.3.1.60Google ScholarDigital Library
- Delone W.H., McLean E.R., 2003. Information Systems Success Revisited. The 35th Hawaii International Conference on System Sciences. DOI: 10.1109/HICSS.2002.994345Google Scholar
- DeLone, W. H., & McLean, E. R. 2016. Information systems success measurement. Foundations and Trends® in Information Systems, 2(1), 1--116. http://dx.doi.org/10.1561/2900000005Google Scholar
- Doll W.J., and Torkzadeh G., 1988. The Measurement of End User Satisfaction. MIS Quarterly, vol.12, n. 2, p. 259--274. https://www.jstor.org/stable/248851Google Scholar
- Doll W.J., and Torkzadeh G., 1998. Developing a Multidimensional Measure of System-Use in an Organizational Context. Information & Management, vol.33, n. 4, p. 171--185. https://doi.org/10.1016/S0378-7206(98)00028-7Google ScholarDigital Library
- Farbey, B., Land, F., and Targett, D. 1993. IT investment: A study of methods and practices. Management Today. Butterworth-Heinemann Ltd., UK.Google Scholar
- Farbey, B., Land, F. and Targett, D. 1999. Moving IS evaluation forward: learning themes and research issues. The Journal of Strategic Information Systems8.2: 189--207. https://doi.org/10.1016/S0963-8687(99)00021-9Google ScholarCross Ref
- Goodhue D.L., 1995 Understanding User Evaluation of Information Systems. Management Science, vol.41, n. 12, p. 18--27. https://doi.org/10.1287/mnsc.41.12.1827Google Scholar
- Goodhue,D.L. and Thompson, R. L., 1995 Task-Technology Fit and Individual Performance. MIS Quarterly, vol.19, n. 2, p. 213--236 (1995). https://www.jstor.org/stable/249689Google ScholarDigital Library
- Grover, V., Jeong S.R. and Segars, A.H. 1996, Information Systems Effectiveness: the Construct Space and Patterns of Application. Information & Management, vol.31, n. 4, p. 177--191. https://doi.org/10.1016/S0378-7206(96)01079-8Google ScholarDigital Library
- Henderson J.C. and Venkatraman N. 1993, Strategic Alignment: Leveraging Information Technology for Transforming Organizations. IBM Systems Journal, vol.32, n. 1, p. 415. DOI: 10.1147/SJ.1999.5387096Google ScholarDigital Library
- Hirschheim R. and Smithson S., 1998. Analyzing Information Systems Evaluation: Another Look at an Old Problem. European Journal of Information Systems, vol.7, n. 3, p. 158--174. https://doi.org/10.1057/palgrave.ejis.3000304Google Scholar
- Irani, Z., and Love, P. E., 2002. Developing a frame of reference for ex-ante IT/IS investment evaluation. European Journal of Information Systems, 11(1), 74--82. DOI: https://doi.org/10.1057/palgrave.ejis.3000411Google ScholarCross Ref
- Ives B., Olson M.H. and Baroudi J. J., 1983. The Measurement of User Information Satisfaction. Communications of the ACM, vol.26, n. 10, p. 785--793. Doi: 10.1145/358413.358430Google Scholar
- Jurison J., 1996. The Temporal Nature of IS Benefits: a Longitudinal Study. Information & Management, vol.30, n. 2, p. 75--79. https://doi.org/10.1016/0378-7206(95)00050-XGoogle ScholarDigital Library
- Kettinger W. J. and Lee C.C. 1997, Pragmatic Perspectives on the Measurement of Information Systems Service Quality. MIS Quarterly, vol.21, n. 2, p. 223--240. https://www.jstor.org/stable/249421Google Scholar
- Kohli R. and Grover V. 2008. Business Value of IT: an Essay on Expanding Research Directions to Keep up with the Times. Journal of the Association for Information Systems, vol.9, n. 1, p. 23--39. https://aisel.aisnet.org/jais/vol9/iss1/1Google ScholarCross Ref
- Markus M.L. and Robey D. 1988. Information Technology and Organizational Change: Casual Structure in Theory and Research. Management Science, vol.34, n. 5, p. 583598. https://doi.org/10.1287/mnsc.34.5.583Google Scholar
- Martinsons M. and Davison R. 1999. The Balanced Score Card: a Foundation for the Strategic Management of Information Systems. Decision Support Systems, vol.25, n. 1, p. 71--87. https://doi.org/10.1016/S0167-9236(98)00086-4Google ScholarDigital Library
- Mirani R. and Lederer A., 1998. An Instrument for Assessing the Organizational Benefits of IS Projects. Decision Sciences, vol.29, n. 4, p. 803--838 (1998). https://doi.org/10.1111/j.1540-5915.1998.tb00878.xGoogle Scholar
- Myers M. 1997. Qualitative Research in Information Systems.MISQuarterly, vol.21, n. 2, p. 241--242. http://www.misq.org/misq/downloads/download/editorial/353Google Scholar
- Nelson R.R., Todd P.A. and Wixom B.H., 2005. Antecedents of Information and System Quality: an Empirical Examination Within the Context of Data Warehousing. Journal of Management Information Systems, vol.21, n. 4, p 199--235 (2005). https://doi.org/10.1080/07421222.2005.11045823Google ScholarDigital Library
- Patton M.Q. 2003. Utilization-Focused Evaluation. In: Kellaghan T., Stufflebeam D. L. (eds) International Handbook of Educational Evaluation. Kluwer International Handbooks of Education, vol 9. Springer, Dordrecht. DOI: https://doi.org/10.1007/978-94-010-0309-4_15Google Scholar
- Petter S., Delone W.H. and McLean E.R., 2008. Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships. European Journal of Information Systems, vol.17, n. 3, p. 236--263. https://doi.org/10.1057/ejis.2008.15Google ScholarCross Ref
- Petter, S., DeLone, W. and McLean, E.R., 2012.The past, present, and future of" IS Success". Journal of the Association for Information Systems, 13(5), p.341. https://aisel.aisnet.org/jais/vol13/iss5/2Google ScholarCross Ref
- Rivard, S., Poirier, G., Raymond, L., and Bergeron, F. 1997.Development of a measure to assess the quality of user-developed applications. ACM SIGMIS Database, 28(3), 44--58. Doi: 10.1145/272657.272690Google ScholarDigital Library
- Rossi, P. H., Lipsey, M. W., and Howard, E. Freeman, 2004. Evaluation: A Systematic Approach. Sage Publications. ISBN: 0761908943Google Scholar
- Seddon P.B. 1997. A Respecification and Extension of the Delone and McLean Model of IS Success. Information Systems Research, vol.8, n. 3, p. 240--254. https://doi.org/10.1287/isre.8.3.240Google ScholarDigital Library
- Sethi V. and King W.R., 1994. Development of Measures to Assess the Extent to Which an Information Technology Application Provides Competitive Advantage. Management Science, vol.40, n. 12, p. 1601--1627 (1994). https://doi.org/10.1287/mnsc.40.12.1601Google Scholar
- Soh C. and Markus M.L., 1995. How IT Creates Business Value: a Process Theory Synthesis. Proceedings of the Sixteenth International Conference on Information Systems, Amsterdam, p. 29--41. Doi=10.1.1.88.8687Google Scholar
- Stockdale, R. and Standing, C., 2006. An interpretive approach to evaluating information systems: A content, context, process framework. European journal of operational research, 173(3), pp. 1090--1102. https://doi.org/10.1016/j.ejor.2005.07.006Google Scholar
- Symons V.J. 1991. A Review of Information Systems Evaluation: Content Context and Process. European Journal of Information Systems, vol.1, p. 205--212. https://doi.org/10.1057/ejis.1991.35Google ScholarCross Ref
- Tate, M., Sedera, D., McLean, E. and Burton-Jones, A., 2011. Information systems success research: the "20-year update?" panel report from PACIS, Communications of the Association for Information Systems, 34, pp. 1235--1246. https://aisel.aisnet.org/cgi/viewcontent.cgi?article=3788&context=caisGoogle Scholar
- Torkzadeh G. and Doll W.J., 1999. The Development of a Tool for Measuring the Perceived Impact of Information Technology on Work. Omega - The International Journal of Management Science, vol.27, n. 3, p. 327--339 https://doi.org/10.1016/S0305-0483(98)00049-8Google ScholarCross Ref
- Urbach, N., Smolnik, S., and Riempp, G. 2009.The state of research on information systems success. Business & Information Systems Engineering, 1(4), 315--325. DOI: https://doi.org/10.1007/s12599-009-0059-yGoogle ScholarCross Ref
- Venkatesh V. 2000. Determinants of Perceived Ease of Use: Integrating Perceived Behavioral Control, Computer Anxiety and Enjoyment into the Technology Acceptance Model. Information Systems Research, vol.11, p. 342--365.Google ScholarDigital Library
- Venkatesh V. and Bala H. 2008. Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences, vol.39, n. 2, p. 273--315. https://doi.org/10.1111/j.1540-5915.2008.00192.xGoogle Scholar
- Venkatesh, V., and Davis, F. D. 1996.A model of the antecedents of perceived ease of use: Development and test. Decision sciences, 27(3), 451--481. https://doi.org/10.1111/j.1540-5915.1996.tb00860.xGoogle Scholar
- Venkatesh V. and Davis F.D. 2000.Theorical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science, vol.46, n. 2, p. 186--204. https://doi.org/10.1287/mnsc.46.2.186.11926Google Scholar
- Willcocks, L. 1992, Evaluating information technology investments: research findings and reappraisal. Information Systems Journal 2, no. 4: 243--268. https://doi.org/10.1111/j.1365-2575.1992.tb00081.xGoogle ScholarCross Ref
- Wixom B.H. and Todd P.A. 2005. A Theorical Integration of User Satisfaction and Technology Acceptance. Information Systems Research, vol.16, n. 1, p. 85--102.Google Scholar
Index Terms
- A Toolbox for Information System Evaluation
Recommendations
Providing a holistic perspective to evaluating e-service quality in web-based systems: a multi-stakeholder approach
SAICSIT '07: Proceedings of the 2007 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countriesOver a period spanning approximately twenty five years IS researchers have been plagued by the problem of how to evaluate IS effectiveness or success. After the advent of the WWW in the 1990s, questions have arisen regarding the relevance of previously ...
Measuring organizational IS effectiveness: an overview and update of senior management perspectives
This paper reports the views of 80 senior IT managers about IT evaluation approaches, and the benefits that IT provides for their organizations. Their views were obtained through a survey mailed to medium to large organizations in both Europe and the ...
IT service management evaluation method based on content, context, and process approach: A literature review
AbstractThe main purpose of implementing Information Technology Service Management (ITSM) is to accomplish high service quality. ITSM evaluation is not an easy matter because of its scale and complexity. Choosing the right evaluation method is essential, ...
Comments