skip to main content
10.1145/3368308.3415402acmconferencesArticle/Chapter ViewAbstractPublication PagesiteConference Proceedingsconference-collections
research-article

ITEval: A Framework for Information Technology Evaluation

Published:07 October 2020Publication History

ABSTRACT

We present a framework of an integrated model, referenced as ITEval, to comprehensively assess and quantitatively compare products of information technology. In this model, we assume that the adoption of technology by an organization is determined by user's willingness to use it, the total cost to own it and its environmental influence. Approaches to quantifying these factors are discussed. Different methods are proposed to analyze and integrate these determinants for assessment and comparison. An example study is employed to demonstrate its application.

References

  1. Dennis A. Adams, R. Ryan Nelson, and Peter A. Todd. 1992. Perceived Usefulness, Ease of Use, and Usage of Information Technology: a Replication. MIS Quarterly 16, 2 (1992), 227--247.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Jesse S. Aronson. 2008. Making IT a Positive Force in Environmental Change. IT Professional 10, 1 (2008), 43--45. https://doi.org/10.1109/MITP.2008.13Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. William Bowen. 1989. The Puny Payoff from Office Computers. MIT Press, Cambridge, MA, USA, 267--271.Google ScholarGoogle Scholar
  4. Alina M. Chircu and Robert J. Kauffman. 2000. Limits to Value in Electronic Commerce-Related IT Investments. J. Manage. Inf. Syst. 17, 2 (2000), 59--80.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Stephen Clement, David McKee, and Jie Xu. 2017. A Service-Oriented CoSimulation: Holistic Data Center Modelling Using Thermal, Power and Computational Simulations. In Proceedings of the10th International Conference on Utility and Cloud Computing (UCC '17). ACM, New York, NY, USA, 91--99. https://doi.org/10.1145/3147213.3147219Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Jessica Colnago, Summer Devlin, Maggie Oates, Chelse Swoopes, Lujo Bauer, Lorrie Cranor, and Nicolas Christin. 2018. 'It's Not Actually That Horrible?: Exploring Adoption of Two-Factor Authentication at a University. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 456, 11 pages. https://doi.org/10.1145/3173574. 3174030Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Julie Smith David, David Schuff, and Robert St. Louis. 2002. Managing your total IT cost of ownership. Commun. ACM 45, 1 (2002), 101--106. https://doi.org/10. 1145/502269.502273Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Fred D. Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13, 3 (1989), 319--340.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Fred D. Davis. 1993. User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies 38, 3 (1993), 475--487.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Fred D. Davis, Richard P. Bagozzi, and Paul R. Warshaw. 1989. User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science 35, 8 (1989), 982--1003. https://doi.org/10.1287/mnsc.35.8.982 arXiv:http://mansci.journal.informs.org/cgi/reprint/35/8/982.pdfGoogle ScholarGoogle ScholarCross RefCross Ref
  11. Brian Desmond, Joe Richards, Robbie Allen, and Alistair G. Lowe-Norris. 2013. Active Directory: Designing, Deploying, and Running Active Directory (5th ed.). O'Reilly Media, Sebastopol, CA, USA.Google ScholarGoogle Scholar
  12. Lisa Diamond, Marc Busch, Valentin Jilch, and Manfred Tscheligi. 2018. Using Technology Acceptance Models for Product Development: Case Study of a Smart Payment Card. In Proceedings of the 20th International Conference on HumanComputer Interaction with Mobile Devices and Services Adjunct (MobileHCI '18). ACM, New York, NY, USA, 400--409. https://doi.org/10.1145/3236112.3236175Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. G. M. U. Din and A. K. Marnerides. 2017. Short term power load forecasting using Deep Neural Networks. In 2017 International Conference on Computing, Networking and Communications (ICNC). IEEE, New York, NY, USA, 594--598. https://doi.org/10.1109/ICCNC.2017.7876196Google ScholarGoogle Scholar
  14. Mazen El-masri and Ali Tarhini. 2017. Factors affecting the adoption of e-learning systems in Qatar and USA: Extending the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2). Educational Technology, Research and Development 65, 3 (06 2017), 743--763. http://search.proquest.com.ezproxy.gvsu.edu/docview/ 1899721379?accountid=39473 Copyright - Educational Technology Research and Development is a copyright of Springer, 2017; Last updated - 2017-05-18; SubjectsTermNotLitGenreText - United States--US; Qatar.Google ScholarGoogle Scholar
  15. Lisa M. Ellram and Sue P. Siferd. 1998. Total cost of ownership: A key concept in strategic cost management decisions. Journal of Business Logistics 19, 1 (1998), 55--84.Google ScholarGoogle Scholar
  16. Nicholas Faulkner, Bradley Jorgensen, and Georgina Koufariotis. 2019. Can behavioural interventions increase citizens? use of e-government? Evidence from a quasi-experimental trial. Government Information Quarterly 36, 1 (2019), 61--68.Google ScholarGoogle ScholarCross RefCross Ref
  17. Bruce G. Ferrin and Richard E. Plank. 2002. Total cost of ownership models: An exploratory study. Journal of Supply Chain Management 38, 3 (2002), 18--29.Google ScholarGoogle ScholarCross RefCross Ref
  18. R. Gitzel, C. Cuske, and C. Munch. 2011. Preliminary Thoughts on Cost-based Investment Decisions in IT: A Problem Analysis. In 2011 Sixth International Conference on Availability, Reliability and Security. IEEE, New York, NY, USA, 385--389. https://doi.org/10.1109/ARES.2011.64Google ScholarGoogle Scholar
  19. E. E. Grandon, A. A. Ibarra, S. A. Guzman, P. Ramirez-Correa, and J. Alfaro-Perez. 2018. Internet of Things: Factors that influence its adoption among Chilean SMEs. In 2018 13th Iberian Conference on Information Systems and Technologies (CISTI). IEEE, New York, NY, USA, 1--6. https://doi.org/10.23919/CISTI.2018.8399183Google ScholarGoogle ScholarCross RefCross Ref
  20. R. Haerting, M. Moehring, R. Schmidt, C. Reichstein, and B. Keller. 2016. What Drives Users to Use CRM in a Public Cloud Environment? - Insights from European Experts. In 2016 49th Hawaii International Conference on System Sciences (HICSS). IEEE, New York, NY, USA, 3999--4008. https://doi.org/10.1109/HICSS. 2016.495Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. S. Han, T. Ravichandran, and J. Kuruzovich. 2010. Competing through Services: Service Migration of Information Technology Product Vendors. In 2010 43rd Hawaii International Conference on System Sciences. IEEE, New York, NY, USA, 1--10. https://doi.org/10.1109/HICSS.2010.113Google ScholarGoogle ScholarCross RefCross Ref
  22. C. Heghedus, A. Chakravorty, and C. Rong. 2018. Energy Load Forecasting Using Deep Learning. In 2018 IEEE International Conference on Energy Internet (ICEI). IEEE, New York, NY, USA, 146--151. https://doi.org/10.1109/ICEI.2018.00-23Google ScholarGoogle Scholar
  23. Ron Henderson and Megan J. Divett. 2003. Perceived usefulness, ease of use and electronic supermarket use. International Journal of Human-Computer Studies 59, 3 (2003), 383--395.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Johan Högberg, Juho Hamari, and Erik Wästlund. 2019. Gameful Experience Questionnaire (GAMEFULQUEST): an instrument for measuring the perceived gamefulness of system use. User Modeling and User-Adapted Interaction 29 (28 Feb 2019), 619--660. https://doi.org/10.1007/s11257-019-09223-wGoogle ScholarGoogle Scholar
  25. Thomas Janicki and Jens O. Liegle. 2001. Development and evaluation of a framework for creating web-based learning modules: a pedagogical and systems perspective. Journal of Asynchronous Learning Networks 5, 1 (2001), 58--84.Google ScholarGoogle Scholar
  26. J. Järvinen, R. Ohtonen, and H. Karjaluoto. 2016. Consumer Acceptance and Use of Instagram. In 2016 49th Hawaii International Conference on System Sciences (HICSS). IEEE, New York, NY, USA, 2227--2236. https://doi.org/10.1109/HICSS.2016.279Google ScholarGoogle Scholar
  27. Jonna Koivisto and Juho Hamari. 2019. The rise of motivational information systems: A review of gamification research. International Journal of Information Management 45 (2019), 191--210.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Patrick Kurp. 2008. Green computing. Commun. ACM 51, 10 (2008), 11--13. https://doi.org/10.1145/1400181.1400186Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. L. Lavazza. 2007. Beyond Total Cost of Ownership: Applying Balanced Scorecards to Open-Source Software. In International Conference on Software Engineering Advances (ICSEA 2007). IEEE, New York, NY, USA, 74--74. https://doi.org/10. 1109/ICSEA.2007.19Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Albert L. Lederer, Donna J. Maupin, Mark P. Sena, and Youlong Zhuang. 2000. The technology acceptance model and the World Wide Web. Decision Support Systems 29, 3 (2000), 269--282. https://doi.org/10.1016/S0167-9236(00)00076-2Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Kimfong Lei and Phillip T. Rawles. 2003. Strategic decisions on technology selections for facilitating a network/systems laboratory using real options & total cost of ownership theories. In CITC4 '03: Proceedings of the 4th conference on Information technology curriculum. ACM, New York, NY, USA, 76--92. https: //doi.org/10.1145/947121.947139Google ScholarGoogle Scholar
  32. Kimfong Lei and Phillip T. Rawles. 2003. Strategic Decisions on Technology Selections for Facilitating a Network/Systems Laboratory Using Real Options & Total Cost of Ownership Theories. In Proceedings of the 4th Conference on Information Technology Curriculum (CITC4 '03). ACM, New York, NY, USA, 76--92. https://doi.org/10.1145/947121.947139Google ScholarGoogle Scholar
  33. Lihui Lin. 2006. Impact of Users? Expertise on the Competition between Proprietary and Open Source Software. In Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06), Vol. 8. IEEE, New York, NY, USA, 166a--166a. https://doi.org/10.1109/HICSS.2006.213Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. W. Long, L. Yuqing, and X. Qingxin. 2013. Using CloudSim to Model and Simulate Cloud Computing Environment. In 2013 Ninth International Conference on Computational Intelligence and Security. IEEE, New York, NY, USA, 323--328. https://doi.org/10.1109/CIS.2013.75Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. B. Martens, M. Walterbusch, and F. Teuteberg. 2012. Costing of Cloud Computing Services: A Total Cost of Ownership Approach. In 2012 45th Hawaii International Conference on System Sciences. IEEE, New York, NY, USA, 1563--1572. https: //doi.org/10.1109/HICSS.2012.186Google ScholarGoogle Scholar
  36. Jaroslaw Milczarek, Piotr Cyplik, and Sebastian Wieczerniak. 2018. USING TOTAL COST OF OWNERSHIP AS A METHOD FOR IDENTIFICATION OF INTERNAL PROBLEMS IN PURCHASE AREA -- CASE STUDY. In Proceedings of The 18th International Scientific Conference Business Logistics in Modern Management. Josip Juraj Strossmayer University of Osijek, Trg Ljudevita Gaja 7, 31000 Osijek, Croatia, 205--223.Google ScholarGoogle Scholar
  37. San Murugesan. 2008. Harnessing Green IT: Principles and Practices. IT Professional 10, 1 (2008), 24--33. https://doi.org/10.1109/MITP.2008.10Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. I.C. Nnorom and O. Osibanjo. 2008. Overview of electronic waste (e-waste) management practices and legislations, and their poor applications in the developing countries. Resources, Conservation and Recycling 52, 6 (2008), 843--858. https://doi.org/10.1016/j.resconrec.2008.01.004Google ScholarGoogle ScholarCross RefCross Ref
  39. Abu Saleh Md Noman, Sanchari Das, and Sameer Patil. 2019. Techies Against Facebook: Understanding Negative Sentiment Toward Facebook via User Generated Content. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, Article 468, 15 pages. https://doi.org/10.1145/3290605.3300698Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. William Panek. 2018. MCSA Windows Server 2016 Complete Study Guide (2nd ed.). Sybex, Hoboken, NJ, USA.Google ScholarGoogle Scholar
  41. V. K. Pant, J. Prakash, and A. Asthana. 2015. Three step data security model for cloud computing based on RSA and steganography. In 2015 International Conference on Green Computing and Internet of Things (ICGCIoT). IEEE, New York, NY, USA, 490--494. https://doi.org/10.1109/ICGCIoT.2015.7380514Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Graham Pickren. 2014. Geographies of E-waste: Towards a Political Ecology Approach to E-waste and Digital Technologies. Geography Compass 8, 2 (2014), 111--124. https://doi.org/10.1111/gec3.12115 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/gec3.12115Google ScholarGoogle ScholarCross RefCross Ref
  43. P. Praveena, G. Subramani, B. Mutlu, and M. Gleicher. 2019. Characterizing Input Methods for Human-to-Robot Demonstrations. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, New York, NY, USA, 344--353. https://doi.org/10.1109/HRI.2019.8673310Google ScholarGoogle Scholar
  44. Daniel Russo, Paolo Ciancarini, Tommaso Falasconi, and Massimo Tomasi. 2018. A Meta-Model for Information Systems Quality: A Mixed Study of the Financial Sector. ACM Transactions on Management Information Systems 9, 3, Article 11 (Sept. 2018), 38 pages. https://doi.org/10.1145/3230713Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Daniel Russo, Paolo Ciancarini, Tommaso Falasconi, and Massimo Tomasi. 2018. A Meta-Model for Information Systems Quality: A Mixed Study of the Financial Sector. ACM Trans. Manage. Inf. Syst. 9, 3, Article 11 (Sept. 2018), 38 pages. https://doi.org/10.1145/3230713Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Raafat Saadé and Bouchaib Bahli. 2005. The impact of cognitive absorption on perceived usefulness and perceived ease of use in on-line learning: an extension of the technology acceptance model. Information and Management 42, 2 (2005), 317--327. https://doi.org/10.1016/j.im.2003.12.013Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Mahmud Akhter Shareef, Bhasker Mukerji, Yogesh K. Dwivedi, and Nripendra P. RanacRubinaIslam. 2019. Social media marketing: Comparative effect of advertisement sources. International Journal of Information Management 46 (2019), 58--69.Google ScholarGoogle Scholar
  48. Kuttimani Tamilmani, Nripendra P. Rana, and Yogesh K. Dwivedi. 2020. Consumer Acceptance and Use of Information Technology: A Meta-Analytic Evaluation of UTAUT2. Information Systems Frontiers 21 (2020), 1--24. https://doi.org/10.1007/ s10796-020-10007-6Google ScholarGoogle Scholar
  49. Kuttimani Tamilmani, Nripendra P. Rana, Naveena Prakasam, and Yogesh K. Dwivedi. 2019. The battle of Brain vs. Heart: A literature review and meta-analysis of 'hedonic motivation' use in UTAUT2. International Journal of Information Management 46 (2019), 222--235. https://doi.org/10.1016/j.ijinfomgt.2019.01.008Google ScholarGoogle ScholarCross RefCross Ref
  50. Viswanath Venkatesh. 1999. Creation of Favorable User Perceptions: Exploring the Role of Intrinsic Motivation. MIS Q. 23, 2 (June 1999), 239--260. https: //doi.org/10.2307/249753Google ScholarGoogle Scholar
  51. Viswanath Venkatesh. 2000. Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, and Emotion into the Technology Acceptance Model. Information Systems Research 11, 4 (2000), 342--365. https://doi.org/10. 1287/isre.11.4.342.11872Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Viswanath Venkatesh and Fred D. Davis. 2000. A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Manage. Sci. 46, 2 (Feb. 2000), 186--204. https://doi.org/10.1287/mnsc.46.2.186.11926Google ScholarGoogle Scholar
  53. Viswanath Venkatesh, Michael G. Morris, Gordon B. Davis, and Fred D. Davis. 2003. User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly 27, 3 (Sept. 2003), 425--478. http://dl.acm.org/citation.cfm?id=2017197. 2017202Google ScholarGoogle ScholarCross RefCross Ref
  54. Viswanath Venkatesh, James Y. L. Thong, and Xin Xu. 2012. Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS Quarterly 36, 1 (2012), 157--178. http://www.jstor.org/stable/41410412Google ScholarGoogle ScholarCross RefCross Ref
  55. Viswanath Venkatesh, James Y. L. Thong, and Xin Xu. 2012. Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS Quarterly 36, 1 (March 2012), 157--178. http://dl.acm.org/citation.cfm?id=2208955.2208966Google ScholarGoogle ScholarCross RefCross Ref
  56. Jen-Her Wu and Shu-Ching Wang. 2005. What drives mobile commerce? An empirical evaluation of the revised technology acceptance model. Information and Management 42, 5 (2005), 719--729. https://doi.org/10.1016/j.im.2004.07.001Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Y. Xiao, R. Bhaumik, Z. Yang, M. Siekkinen, P. Savolainen, and A. Yla-Jaaski. 2010. A System-Level Model for Run-time Power Estimation on Mobile Devices. In 2010 IEEE/ACM Int'l Conference on Green Computing and Communications Int'l Conference on Cyber, Physical and Social Computing. IEEE, New York, NY, USA, 27--34. https://doi.org/10.1109/GreenCom-CPSCom.2010.114Google ScholarGoogle Scholar
  58. A. S. Yahaya, N. Javaid, K. Latif, and A. Rehman. 2019. An Enhanced Very Short-Term Load Forecasting Scheme Based on Activation Function. In 2019 International Conference on Computer and Information Sciences (ICCIS). IEEE, New York, NY, USA, 1--6. https://doi.org/10.1109/ICCISci.2019.8716384Google ScholarGoogle Scholar

Index Terms

  1. ITEval: A Framework for Information Technology Evaluation

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        SIGITE '20: Proceedings of the 21st Annual Conference on Information Technology Education
        October 2020
        446 pages
        ISBN:9781450370455
        DOI:10.1145/3368308

        Copyright © 2020 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 7 October 2020

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

        Acceptance Rates

        Overall Acceptance Rate176of429submissions,41%
      • Article Metrics

        • Downloads (Last 12 months)7
        • Downloads (Last 6 weeks)0

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader