Abstract
The body of literature in the area of IT/IS investment evaluation proposes a variety of methods and techniques for the ex-ante assessment of IT/IS investments. However, the financial evaluation of intangible benefits associated with IT/IS investments in general and investments in innovative IT/IS in particular still remains a problem area of high relevance to decision makers. As a consequence, investment decisions are still taken by practitioners as an “act of faith”. With this in mind, it is our primary aim to develop a quantification model for the financial assessment of intangible benefits concerning investments in innovative IT/IS. Based on an augmented reality and smart glasses application scenario from the construction domain, we demonstrate how intangible benefits of innovative IT/IS investments can be visualised and measured more effectively by means of utility effect chains and system dynamics prior to their incorporation into a cost–benefit analysis. Based on design science research, the quantification model is developed by means of a systematic literature review and evaluated using the augmented reality application scenario in construction. The model can serve as an aid to the managerial decision making process by providing an enhanced understanding of the various tangible and intangible benefits associated with the investment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The “technology readiness level” (TRL) scale has been introduced by the NASA in the 1970s to describe the maturity of a technology (NASA 2015).
References
Agarwal S (2016) Review on application of augmented reality in civil engineering. In: Proceedings of the international conference on inter disciplinary research in engineering and technology 2016. New Delhi, India, pp 68–71
Algohary T (2015) Automating construction progress monitoring and communication. BIM Hub, Dubai
Alpar P, Alt R, Bensberg F et al (2014) Planung und Steuerung des Einsatzes von IS. In: Anwendungsorientierte Wirtschaftsinformatik. Springer, Fachmedien, Wiesbaden, pp 39–72
Anandarajan A, Wen HJ (1999) Evaluation of information technology investment. Manag Decis 37:329–339. https://doi.org/10.1108/00251749910269375
Anselstetter DR (1984) Nutzeffekte der DV in den Leistungsbereichen von Unternehmen. Betriebswirtschaftliche Nutzeffekte der Datenverarbeitung. Springer, Berlin, pp 78–243
Bacon CJ (1992) The use of decision criteria in selecting information systems/technology investments. MIS Q 16:335–353. https://doi.org/10.2307/249532
Ballantine J, Stray S (1998) Financial appraisal and the IS/IT investment decision making process. J Inf Technol Routledge Ltd 13:3–14. https://doi.org/10.1080/026839698344927
Baltzer B (2013) Erfolgsbeurteilung. In: Einsatz und Erfolg von Controlling-Instrumenten. Springer, Fachmedien, Wiesbaden, pp 163–233
Bauernhansl T, Hompel M ten, Vogel-Heuser B (2014) Industrie 4.0 in Produktion, Automatisierung und Logistik: Anwendung, Technologien, Migration. Springer
Behzadan AH, Kamat VR (2013) Enabling discovery-based learning in construction using telepresent augmented reality. Autom Constr 33:3–10. https://doi.org/10.1016/j.autcon.2012.09.003
Behzadan AH, Dong S, Kamat VR (2015) Augmented reality visualization: a review of civil infrastructure system applications. Adv Eng Inf 29:252–267. https://doi.org/10.1016/j.aei.2015.03.005
Beitzel S, Dykstra J, Huver S et al (2016) Cognitive performance impact of augmented reality for network operations tasks. Advances in human factors in cybersecurity. Springer, Cham, pp 139–151
Benham J (2015) The evolution of wearable tech in construction industry, part 2. http://jamesbenham.com/the-evolution-of-wearable-tech-in-construction-industry-part-2. Accessed 10 July 2016
Binswanger M (2001) Technological progress and sustainable development: what about the rebound effect? Ecol Econ 36:119–132. https://doi.org/10.1016/S0921-8009(00)00214-7
Bischoff J, Taphorn C, Wolter D et al (2015) Erschliessen der Potenziale der Anwendung von “Industrie 4.0” im Mittelstand. agriplan GmbH
Bitnamic (2017) VIOS AR im check: Die Zukunft der industriellen Instandhaltung? https://bitnamicwordpressapp.azurewebsites.net/blog/de/2017/04/07/vios-ar-im-check-die-zukunft-der-industriellen-instandhaltung/. Accessed 15 Apr 2017
British CIMA, IProdE (1987) Justifying investments in advanced manufacturing projects. Kogan Page, London
Choi JW (2009) A technological review to develop an ar-based design supporting system. In: Wang X, Schnabel MA (eds) Mixed reality in architecture, design and construction. Springer, Netherlands, pp 53–74
Clark TD, Augustine FK (1992) Using system dynamics to measure the value of information in a business firm. Syst Dyn Rev 8:149–173. https://doi.org/10.1002/sdr.4260080204
Cleven A, Gubler P, Hüner KM (2009) Design alternatives for the evaluation of design science research artifacts. In: Proceedings of the 4th international conference on design science research in information systems and technology. ACM, New York, NY, USA, pp 19:1–19:8
Coyle RG (1996) System dynamics modelling: a practical approach. CRC Press, London
Culp S, Sharratt I (2015) The evolution of risk technology. In: Accenture finance & risk services. https://www.accenture.com/t20150714T065457__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/Industries_6/Accenture-The-Evolution-of-Risk-Technology.pdf. Accessed 17 Oct 2016
Danker F, Jones O (2014) Combining augmented reality and building information modelling—an industry perspective on applications and future directions. In: Thompson EM (ed) Fusion—proceedings of the 32nd eCAADe conference—volume 2. Department of Architecture and Built Environment, Faculty of Engineering and Environment, Newcastle Upon Tyne, England, UK, 10–12 September 2014, pp 525–536
DAQRI Smart Helmet (2016) http://daqri.com/home/product/daqri-smart-helmet/. Accessed 8 Jun 2016
Degraeve Z, Labro E, Roodhooft F (2000) An evaluation of vendor selection models from a total cost of ownership perspective. Eur J Oper Res 125:34–58. https://doi.org/10.1016/S0377-2217(99)00199-X
Dewan S, Shi C, Gurbaxani V (2007) Investigating the risk–return relationship of information technology investment: firm-level empirical analysis. Manag Sci 53:1829–1842. https://doi.org/10.1287/mnsc.1070.0739
Dhawan R (2005) Usefulness of probabilistic system dynamics in dynamic decision making. In: Proceedings of the 23rd international conference of the system dynamics society, Boston, USA
Dos Santos BL, Peffers K (1995) Rewards to investors in innovative information technology applications: first movers and early followers in ATMs. Organ Sci 6:241–259. https://doi.org/10.1287/orsc.6.3.241
Dos Santos BL, Peffers K, Mauer DC (1993) The impact of information technology investment announcements on the market value of the firm. Inf Syst Res 4:1–23. https://doi.org/10.1287/isre.4.1.1
Dunston PS, Wang X (2005) Mixed reality-based visualization interfaces for architecture, engineering, and construction industry. J Constr Eng Manag 131:1301–1309. https://doi.org/10.1061/(ASCE)0733-9364(2005)131:12(1301)
Dzwiarek M, Luczak A (2008) Application prospects of the augmented reality technology for improving safety of machine operators. In: Human–computer interaction: new developments
Erkoyuncu JA, del Amo IF, Dalle Mura M et al (2017) Improving efficiency of industrial maintenance with context aware adaptive authoring in augmented reality. CIRP Ann 66:465–468. https://doi.org/10.1016/j.cirp.2017.04.006
Ervin R (2016) These 5 virtual reality headsets could revolutionize field service. In: Field Serv. Digit. http://fsd.servicemax.com/2016/02/19/5-virtual-reality-headsets-revolutionize-field-service/. Accessed 10 July 2016
Fridgen G, Moser F (2013) Using IT fashion investments to optimize an IT innovation portfolio’s risk and return. J Decis Syst 22:298–318. https://doi.org/10.1080/12460125.2013.835507
Friedman E (2015) Wearable technology by industry series: vol. 5—construction. In: Enterp. Blog. https://brainxchange.events/blog/wearable-technology-industry-series-construction/. Accessed 10 July 2016
Gordon TJ, Stover J (1976) Using perceptions and data about the future to improve the simulation of complex systems. Technol Forecast Soc Change 9:191–211. https://doi.org/10.1016/0040-1625(76)90049-4
Gossart C (2015) Rebound effects and ICT: a review of the literature. In: Hilty LM, Aebischer B (eds) ICT innovations for sustainability. Springer, Berlin, pp 435–448
Götze U, Northcott D, Schuster P (2015) Investment appraisal: methods and models. Springer, Berlin
Grob HL (1989) Investitionsrechnung mit vollständigen Finanzplänen, 1st edn. Vahlen, München
Häckel B, Isakovic V, Moser F (2015) Integrated long- and short-term valuation of IT innovation investments. Electron Mark 25:73–85. https://doi.org/10.1007/s12525-014-0171-9
Hänsch F (2015) Finanzwirtschaftliche ex-ante Bewertung intangibler Benefits von IT-Projekten. HMD Prax Wirtsch 52:945–957. https://doi.org/10.1365/s40702-015-0187-4
Hares JS, Royle D (1994) Measuring the value of information technology. Wiley, Chichester
Hertwich EG (2005) Consumption and the rebound effect: an industrial ecology perspective. J Ind Ecol 9:85–98. https://doi.org/10.1162/1088198054084635
Hevner AR, March ST, Park J, Ram S (2004) Design science in information systems research. MIS Q 28:75–105
Hilty LM, Köhler A, Schéele FV et al (2006) Rebound effects of progress in information technology. Poiesis Prax 4:19–38. https://doi.org/10.1007/s10202-005-0011-2
Hochstrasser B (1990) Evaluating IT investments—matching techniques to projects. J Inf Technol 5:215–221. https://doi.org/10.1057/jit.1990.45
Hofmann B, Haustein D, Landeweerd L (2017) Smart-glasses: exposing and elucidating the ethical issues. Sci Eng Ethics 23:701–721. https://doi.org/10.1007/s11948-016-9792-z
Hörning KH, Ahrens D, Gerhard A (1999) Do technologies have time? New practices of time and the transformation of communication technologies. Time Soc 8:293–308. https://doi.org/10.1177/0961463X99008002005
Huang X, Kunc M (2012) Modeling start-ups using system dynamics: towards a generic model. In: Annals do international conference of the system dynamics society, St. Gallen, Switzerland
Irani Z (2002) Information systems evaluation: navigating through the problem domain. Inf Manag 40:11–24. https://doi.org/10.1016/S0378-7206(01)00128-8
Irani Z, Love PED (2000) The propagation of technology management taxonomies for evaluating investments in information systems. J Manag Inf Syst 17:161–177. https://doi.org/10.1080/07421222.2000.11045650
Irani Z, Love P (2002) Developing a frame of reference for ex-ante IT/IS investment evaluation. Eur J Inf Syst 11:74
Irani Z, Ghoneim A, Love PED (2006) Evaluating cost taxonomies for information systems management. Eur J Oper Res 173:1103–1122. https://doi.org/10.1016/j.ejor.2005.07.007
Jede A, Teuteberg F (2016) Investigating preconditions for a financially advantageous cloud usage. Int J Account Inf Manag 24:116–134. https://doi.org/10.1108/IJAIM-04-2015-0018
Jiao Y, Zhang S, Li Y et al (2013) Towards cloud augmented reality for construction application by BIM and SNS integration. Autom Constr 33:37–47. https://doi.org/10.1016/j.autcon.2012.09.018
Jones K (2014) Five ways the construction industry will benefit from augmented reality. In: LinkedIn Pulse. https://www.linkedin.com/pulse/20140805163603-41493855-five-ways-the-construction-industry-will-benefit-from-augmented-reality. Accessed 30 May 2016
Kagermann H, Helbig J, Hellinger A, Wahlster W (2013) Recommendations for implementing the strategic initiative INDUSTRIE 4.0: securing the future of German manufacturing industry; final report of the Industrie 4.0 Working Group. Forschungsunion
Kahen G, Lehman M, Ramil J, Wernick P (2000) Dynamic modelling in the investigation of policies for E-type software evolution. In: Proceedings of the software process simulation modeling workshop (ProSim 2000)
Kamat V, Martinez J, Fischer M et al (2010) Research in visualization techniques for field construction. J Constr Eng Manag 137:853–862
Kennedy M (1999) Some issues in building system dynamics models designed to improve the information systems investment appraisal. In: Proceedings of the 17th international conference of the system dynamics society, Wellington, New Zealand
Kim JH, Chan T, Du W (2015) The learning effect of augmented reality training in a computer-based simulation environment. Learning and collaboration technologies. Springer, Cham, pp 406–414
Kim S, Nussbaum MA, Gabbard JL (2016) Augmented reality “smart glasses” in the workplace: industry perspectives and challenges for worker safety and health. IIE Trans Occup Ergon Hum Factors 4:253–258. https://doi.org/10.1080/21577323.2016.1214635
Kivrak S, Arslan G, Akgun A, Arslan V (2013) Augmented reality system applications in construction project activities. In: ISARC Proc 2013 Proceedings of the 30th ISARC, Montréal, Canada, pp 1560–1571
Lederer AL, Prasad J (1993) Information systems software cost estimating: a current assessment. J Inf Technol 8:22–33. https://doi.org/10.1057/jit.1993.4
Lichtblau K, Stich V, Bertenrath R et al (2015) IMPULS-Industrie 4.0-Readiness. Impuls-Stiftung des VDMA, Aachen-Köln
Limsupreeyarat P, Yabuki N, Tongthong T (2010) Toward the improvement of safety planning for construction activities performed at high elevation by using augmented reality. In: Proceedings of the international conference on computing in civil and building engineering, p 9
Malone J (2015) Technology leads the way. Constr Eur 26:42–47
March ST, Storey VC (2008) Design science in the information systems discipline: an introduction to the special issue on design science research. MIS Q 32:725–730
Matlin G (1979) What is the value of investment in information systems? MIS Q 3:5–34. https://doi.org/10.2307/248786
Melville N, Kraemer K, Gurbaxani V (2004) Review: information technology and organizational performance: an integrative model of it business value. MIS Q 28:283–322
Miah T, Carter C, Thorpe A et al (1998) Wearable computers—an application of BT’s mobile video system for the construction industry. BT Technol J 16:191–199. https://doi.org/10.1023/A:1009645327332
Milis K, Mercken R (2004) The use of the balanced scorecard for the evaluation of information and communication technology projects. Int J Proj Manag 22:87–97. https://doi.org/10.1016/S0263-7863(03)00060-7
Moser F (2011) Evaluating IT fashion investments regarding risk and return. In: AMCIS. Citeseer, Detroit, Michigan
Murphy KE, Simon SJ (2002) Intangible benefits valuation in ERP projects. Inf Syst J 12:301–320. https://doi.org/10.1046/j.1365-2575.2002.00131.x
Nair GK, Rodrigues LLR (2013) Dynamics of financial system: a system dynamics approach. Int J Econ Financ Issues 3:14–26
NASA (2015) Technology readiness level. http://www.nasa.gov/directorates/heo/scan/engineering/technology/txt_accordion1.html. Accessed 17 Oct 2016
Nerney C (2016) Augmented reality, wearable computing offer big benefits. In: Comput. Sci. Corp. http://www.csc.com/infrastructure_services/insights/101025-augmented_reality_wearable_computing_offer_big_benefits. Accessed 10 July 2016
Nielsen S, Nielsen EH (2008) System dynamics modelling for a balanced scorecard: computing the influence of skills, customers, and work in process on the return on capital employed. Manag Res News 31:169–188. https://doi.org/10.1108/01409170810851276
O’Halloran D, Kvochko E, Daugherty P et al (2015) Industrial internet of things: unleashing the potential of connected products and services. World Economic Forum, Cologny
Ordoobadi SM (2012) Application of AHP and Taguchi loss functions in evaluation of advanced manufacturing technologies. Int J Adv Manuf Technol 67:2593–2605. https://doi.org/10.1007/s00170-012-4676-0
Ozturan M, Basarir-Ozel B, Akar E (2016) A review on methods for the assessment of information system projects. J Inf Technol Appl. https://doi.org/10.7251/JIT1502117O
Pasher E, Popper Z, Raz H, Lawo M (2010) WearIT@work: a wearable computing solution for knowledge-based development. Int J Knowl Based Dev 1:346–360
PAULEY (2015) The future of mobile is augmented. In: PAULEY Blog. http://www.pauley.co.uk/blog/the-future-of-mobile-is-augmented/. Accessed 10 July 2016
Pierdicca R, Frontoni E, Pollini R et al (2017) The use of augmented reality glasses for the application in industry 4.0. In: Augmented reality, virtual reality, and computer graphics. Springer, Cham, pp 389–401
Piotrowicz W, Irani Z (2010) Analysing B2B electronic procurement benefits: information systems perspective. J Enterp Inf Manag 23:559–579. https://doi.org/10.1108/17410391011061816
Pries-Heje J, Baskerville R, Venable J (2008) Strategies for design science research evaluation. ECIS 2008 Proc
Primrose PL (1990) Selecting and evaluating cost-effective MRP and MRPII. Int J Oper Prod Manag 10:51–66. https://doi.org/10.1108/01443579010141794
Renkema TJW, Berghout EW (1997) Methodologies for information systems investment evaluation at the proposal stage: a comparative review. Inf Softw Technol 39:1–13. https://doi.org/10.1016/0950-5849(96)85006-3
Rubenstone J (2015) Putting on tomorrow’s work gear. (Cover story). ENR Eng News-Rec 275:24–29
Sassone PG, Schaffer WA (1978) Cost–benefit analysis: a handbook. Academic Press, London
Schnabel MA (2009) Framing mixed realities. In: Wang X, Schnabel MA (eds) Mixed reality in architecture, design and construction. Springer, Netherlands, pp 3–11
Schryen G (2010) Preserving knowledge on is business value. Bus Inf Syst Eng 2:233–244. https://doi.org/10.1007/s12599-010-0111-y
Schumann M, Linß H (1993) Wirtschaftlichkeitsbeurteilung von DV-Projekten. Informationsmanagement. Gabler Verlag, Wiesbaden, pp 69–92
Schwarz R, Schoeneborn F (2002) An elementary dynamic model of a small start-up firm. In: Proceedings of the 20th international conference of the system dynamics society. Palermo
Shang S, Seddon PB (2002) Assessing and managing the benefits of enterprise systems: the business manager’s perspective. Inf Syst J 12:271–299. https://doi.org/10.1046/j.1365-2575.2002.00132.x
Silva LCE, Costa APCS (2014) IT project investments: an analysis based on a sort and rank problem. Int J Inf Technol Decis Mak 13:699–719. https://doi.org/10.1142/S0219622014500655
Skibniewski MJ (2014) Mobile and pervasive computing in construction. Constr Manag Econ 32:1148–1150. https://doi.org/10.1080/01446193.2014.949810
Sonnenberg C, vom Brocke J (2011) Evaluation patterns for design science research artefacts. In: Helfert M, Donnellan B (eds) Practical aspects of design science. Springer, Berlin, pp 71–83
Spath D, Ganschar O, Gerlach S et al (2013) Produktionsarbeit der Zukunft—Industrie 4.0. Fraunhofer-Institut für Arbeitswirtschaft und Organisation, Stuttgart
Sterman JD (2000) Business dynamics. Inklusiv CD: systems thinking and modeling for a complex world, Har/Cdr. Mcgraw-Hill, Boston
Stratopoulos TC, Lim J-H (2010) IT innovation persistence: an oxymoron? Commun ACM 53:142–146. https://doi.org/10.1145/1735223.1735258
Suárez-Warden F, Mendívil EG, Rodríguez CA, Garcia-Lumbreras S (2015) Assembly operations aided by augmented reality: an endeavour toward a comparative analysis. Proc Comput Sci 75:281–290. https://doi.org/10.1016/j.procs.2015.12.249
Syberfeldt A, Danielsson O, Holm M, Wang L (2015) Visual assembling guidance using augmented reality. Proc Manuf 1:98–109. https://doi.org/10.1016/j.promfg.2015.09.068
Tayyari F, Kroll DE (1990) Total cost analysis of modern automated systems. In: Justification methods for integrated manufacturing systems. Elsevier, New York, USA, pp 234–241
Theisa S, Mertens A, Wille M et al (2015) Effects of data glasses on human workload and performance during assembly and disassembly tasks. In: Proceedings 19th triennial congress of the IEA, p 14
Thiel C, Thiel C (2014) Enforcing data privacy in the age of Google glass. In: Reimer H, Pohlmann N, Schneider W (eds) ISSE 2014 securing electronic business processes. Springer, Fachmedien, Wiesbaden, pp 220–229
Van Hampton T, Rubenstone J, Sawyer T (2016) Game changer. ENR Eng News-Rec 28–33
Ventana Systems, Inc. (2017) Vensim® PLE Plus | Vensim. http://vensim.com/vensim-ple-plus/. Accessed 11 Apr 2017
vom Brocke J, Lindner MA (2004) Service Portfolio measurement: a framework for evaluating the financial consequences of out-tasking decisions. In: Proceedings of the 2nd international conference on service oriented computing. ACM, New York, NY, USA, pp 203–211
vom Brocke J, Simons A, Niehaves B et al (2009) Reconstructing the giant: on the importance of rigour in documenting the literature search process. In: Newell S, Whitley E, Pouloudi N et al (eds) Proceedings of the ECIS 2009. Verona, pp 2206–2217
Walter SG, Spitta T (2004) Approaches to the ex-ante evaluation of investments into information systems. Wirtschaftsinformatik 46:171–180. https://doi.org/10.1007/BF03250934
Wang X (2009) Augmented reality in architecture and design: potentials and challenges for application. Int J Archit Comput 7:309–326. https://doi.org/10.1260/147807709788921985
Wang P (2010) Chasing the hottest IT: effects of information technology fashion on organizations. MIS Q 34:63–85
Wang X, Dunston PS (2013) Tangible mixed reality for remote design review: a study understanding user perception and acceptance. Vis Eng 1:1–15. https://doi.org/10.1186/2213-7459-1-8
Wang X, Love PE (2012) BIM + AR: onsite information sharing and communication via advanced visualization. In: 2012 IEEE 16th international conference on computer supported cooperative work in design (CSCWD), pp 850–855
Wang X, Kim MJ, Love PED, Kang S-C (2013a) Augmented reality in built environment: classification and implications for future research. Autom Constr 32:1–13. https://doi.org/10.1016/j.autcon.2012.11.021
Wang X, Love PED, Kim MJ et al (2013b) A conceptual framework for integrating building information modeling with augmented reality. Autom Constr 34:37–44. https://doi.org/10.1016/j.autcon.2012.10.012
Ward J, Taylor P, Bond P (1996) Evaluation and realisation of IS/IT benefits: an empirical study of current practice. Eur J Inf Syst 4:214–225. https://doi.org/10.1057/ejis.1996.3
Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q 26:xiii–xxiii
Willcocks L, Margetts H (1994) Risk assessment and information systems. Eur J Inf Syst 3:127–138. https://doi.org/10.1057/ejis.1994.13
Williams E (2011) Environmental effects of information and communications technologies. Nature 479:354–358. https://doi.org/10.1038/nature10682
Wolstenholme EF, Gavine A, Watts KM, Henderson S (1992) The design, application and evaluation of a system dynamics based methodology for the assessment of computerised information systems. Eur J Inf Syst 1:341–350. https://doi.org/10.1057/ejis.1992.6
Acknowledgements
The authors would like to thank the anonymous reviewers as well as Ms. Imhorst for their valuable help and substantive feedback which have contributed to the improvement of this paper.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Oesterreich, T.D., Teuteberg, F. Why one big picture is worth a thousand numbers: measuring intangible benefits of investments in augmented reality based assistive technology using utility effect chains and system dynamics. Inf Syst E-Bus Manage 16, 407–441 (2018). https://doi.org/10.1007/s10257-017-0367-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10257-017-0367-6