Skip to main content
SpringerLink
Log in

Gamifying the Human-in-the-Loop: Toward Increased Motivation for Training AI in Customer Service

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13336))

Included in the following conference series:

Abstract

In this paper, we contribute to research on human-AI collaboration in the scope of Hybrid Intelligence Systems, which enable mutual augmentation and collaborative learning of both human and AI. Thereby, we address a research gap focusing on the continuance intention of customer service employees to teach AI during their work task. So far, the human-in-the-loop (HITL) approach is commonly applied to directly involve the human user in Machine Learning (ML) to actively advance AI. However, there is only little consideration of users’ motivation regarding the extra effort of teaching AI during their work. To address this gap, we combine gamification and ML toward increased motivation to participate in HITL learning. Therefore, we follow the Design Science Research process and align to a framework for gamifying collaboration processes. Thus, we identify meta-requirements toward intended outcomes of gamified collaboration based on expert interviews, consequently derive design principles to gamify the process interactions and determine matching gamification elements. For demonstration, we implement the design principles with the according gamification elements in a prototype enabling customer service employees to provide feedback to an AI. Eventually, we evaluate the design principles with the prototype in user test simulations. The results reveal their successful implementation as well as the relevance of mixed gamification elements to trigger various motivation types. Additionally, we show applicability of the framework to gamify human-AI collaboration.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Statistik der Woche. https://www.heise.de/news/Statistik-der-Woche-Wo-KI-in-der-deutschen-Wirtschaft-boomt-6172430.html. Accessed 8 Feb 2021

  2. Huang, M.-H., Rust, R.T.: Artificial intelligence in service. J. Serv. Res. 21(2), 155–172 (2018)

    Article  Google Scholar 

  3. de Keyser, A., Köcher, S., Alkire, L., Verbeeck, C., Kandampully, J.: Frontline service technology infusion: conceptual archetypes and future research directions. J. Serv. Manag. 30(1), 156–183 (2019)

    Article  Google Scholar 

  4. Robinson, S., et al.: Frontline encounters of the AI kind: an evolved service encounter framework. J. Bus. Res. 116, 366–376 (2020)

    Article  Google Scholar 

  5. Xu, Y., Shieh, C.-H., van Esch, P., Ling, I.-L.: AI customer service: task complexity, problem-solving ability, and usage intention. Australas. Mark. J. 28(4), 189–199 (2020)

    Article  Google Scholar 

  6. Holzinger, A.: Interactive Machine Learning (iML). Informatik Spektrum 39(1), 64–68 (2016)

    Article  Google Scholar 

  7. Holzinger, A., Plass, M., Holzinger, K., Crişan, G.C., Pintea, C.-M., Palade, V.: Towards interactive Machine Learning (iML): applying ant colony algorithms to solve the traveling salesman problem with the human-in-the-loop approach. In: Buccafurri, F., Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-ARES 2016. LNCS, vol. 9817, pp. 81–95. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-45507-5_6

    Chapter  MATH  Google Scholar 

  8. Amershi, S., Cakmak, M., Knox, W.B., Kulesza, T.: Power to the people: the role of humans in interactive machine learning. AI Mag. 35(4), 105–120 (2014)

    Google Scholar 

  9. Holzinger, A.: Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Inform. 3, 119–131 (2016)

    Article  Google Scholar 

  10. Holzinger, A., Plass, M., Holzinger, K., Crisan, G.C., Pintea, C.-M., Palade, V.: A glass-box interactive machine learning approach for solving NP-hard problems with the human-in-the-loop. arXiv:1708.01104v1 (2017)

  11. Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., Ebel, P.: The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems. In: Proceedings of the 52nd Hawaii International Conference on System Sciences (HICSS), Hawaii, USA (2019)

    Google Scholar 

  12. Dellermann, D., Ebel, P., Söllner, M., Leimeister, J.M.: Hybrid intelligence. Bus. Inf. Syst. Eng. 61, 637–643 (2019)

    Article  Google Scholar 

  13. Bittner, E., Oeste-Reiß, S., Leimeister, J.M.: Where is the bot in our team? Toward a taxonomy of design option combinations for conversational agents in collaborative work. In: Proceedings of the 52nd Hawaii International Conference on System Sciences (HICSS), Hawaii, USA (2019)

    Google Scholar 

  14. Frick, W.: When your boss wears metal pants. Harvard Bus. Rev. 93, 84–89 (2015)

    Google Scholar 

  15. Seeber, I., et al.: Machines as teammates: a research agenda on AI in team collaboration. Inf. Manag. 57, 103174 (2020)

    Google Scholar 

  16. Martínez, M.A.M., Nadj, M., Maedche, A.: Towards an integrative theoretical framework of interactive machine learning systems. In: Proceedings of the 27th European Conference on Information Systems, Stockholm-Uppsala, Sweden (2019)

    Google Scholar 

  17. Dudley, J.J., Kristensson, P.O.: A review of user interface design for interactive machine learning. ACM Trans. Interact. Intell. Syst. 8(1), 1–37 (2018)

    Article  Google Scholar 

  18. Lowry, P.B., Gaskin, J.E., Moody, G.D.: Proposing the Multi-Motive Information Systems Continuance Model (MISC) to better explain end-user system evaluations and continuance intentions. J. Assoc. Inf. Syst. 16(7), 515–579 (2015)

    Google Scholar 

  19. Bhattacherjee, A., Premkumar, G.: Understanding changes in belief and attitude toward information technology usage. A theoretical model and longitudinal test. Manag. Inf. Syst. Q. 28(2), 229–254 (2004)

    Article  Google Scholar 

  20. Darejeh, A., Salim, S.S.: Gamification solutions to enhance software user engagement - a systematic review. Int. J. Hum.-Comput. Interact. 32(8), 613–642 (2016)

    Article  Google Scholar 

  21. Steffens, F., Marczak, S., Figueira Filho, F., Treude, C., Singer, L., Redmiles Ban Al-Ani, D.: Using gamification as a collaboration motivator for software development teams. A preliminary framework. In: Brazilian Symposium on Collaborative Systems, Salvador, Brazil (2015)

    Google Scholar 

  22. Khakpour, A., Colomo-Palacios, R.: Convergence of gamification and machine learning: a systematic literature review. Technol. Learn. 26, 597–636 (2021)

    Google Scholar 

  23. Meske, C., Brockmann, T., Wilms, K., Stieglitz, S.: Gamify employee collaboration - a critical review of gamification elements in social software. In: Australasian Conference on Information Systems, Adelaide (2016)

    Google Scholar 

  24. Wiethof, C., Tavanapour, N., Bittner, E.: Designing and evaluating a collaborative writing process with gamification elements: toward a framework for gamifying collaboration processes. AIS Trans. Hum.-Comput. Interact. 13(1), 38–61 (2021)

    Article  Google Scholar 

  25. Gregor, S.: The nature of theory in information systems. MIS Q. 30(3), 611–642 (2006)

    Article  Google Scholar 

  26. Gregor, S., Hevner, A.: Positioning and presenting design science research for maximum impact. MIS Q. 37(2), 337–355 (2013)

    Article  Google Scholar 

  27. Peffers, K., et al.: The design science research process. A model for producing and presenting information systems research. In: Proceedings of the International Conference on Design Science Research in Information Systems and Technology, Claremont, CA (2006)

    Google Scholar 

  28. Cheng, X., Yin, G., Azadegan, A., Kolfschoten, G.: Trust evolvement in hybrid team collaboration: a longitudinal case study. Group Decis. Negot. 25(2), 267–288 (2015). https://doi.org/10.1007/s10726-015-9442-x

    Article  Google Scholar 

  29. Przybilla, L., Baar, L., Wiesche, M., Krcmar, H.: Machines as teammates in creative teams: digital facilitation of the dual pathway to creativity. In: Proceedings of SIGMIS-CPR, Nashville, TN, USA (2019)

    Google Scholar 

  30. Wiethof, C., Tavanapour, N., Bittner, E.: Implementing an intelligent collaborative agent as teammate in collaborative writing: toward a synergy of humans and AI. In: Proceedings of the 54th Hawaii International Conference on System Sciences (2021)

    Google Scholar 

  31. Benbya, H., Pachidi, S., Jarvenpaa, S.L.: Artificial intelligence in organizations: implications for information systems research. J. Assoc. Inf. Syst. 22(2), 281–303 (2021)

    Google Scholar 

  32. Larivière, B., et al.: “Service Encounter 2.0’’: an investigation into the roles of technology, employees and customers. J. Bus. Res. 79, 238–246 (2017)

    Article  Google Scholar 

  33. Wiethof, C., Bittner, E.: Hybrid intelligence - combining the human in the loop with the computer in the loop: a systematic literature review. In: Forty-Second International Conference on Information Systems, Austin (2021)

    Google Scholar 

  34. Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., He, L.: A survey of human-in-the-loop for machine learning. arXiv:2108.00941v1 (2021)

  35. Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From game design elements to gamefulness: defining “gamification”. In: MindTrek 2011, Tampere, Finland (2011)

    Google Scholar 

  36. Hamari, J., Koivisto, J., Sarsa, H.: Does gamification work? - A literature review of empirical studies on gamification. In: Proceedings of the 47th Hawaii International Conference on System Sciences, Hawaii, USA (2014)

    Google Scholar 

  37. Marczak, S., Filho, F.F., Singer, L, Treude, C., Steffens, F., Redmiles Ban Al-Ani, D.: Studying gamification as a collaboration motivator for virtual software teams. Social issues, cultural issues, and research methods. In: Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work and Social Computing, Vancouver, BC, Canada (2015)

    Google Scholar 

  38. Hunicke, R., LeBlanc, M., Zubek, R.: MDA: a formal approach to game design and game research. In: Proceedings of the Challenges in Games AI Workshop, Nineteenth National Conference of Artificial Intelligence, AAAI, San Jose, USA, pp. 1–5 (2004)

    Google Scholar 

  39. Liu, D., Santhanam, R., Webster, J.: Toward meaningful engagement: a framework for design and research of gamified information systems. Manag. Inf. Syst. Q. 41(4), 1011–1034 (2017)

    Article  Google Scholar 

  40. Suh, A., Cheung, C.M.K., Ahuja, M., Wagner, C.: Gamification in the workplace: the central role of the aesthetic experience. J. Manag. Inf. Syst. 34(1), 268–305 (2017)

    Article  Google Scholar 

  41. Tseng, S.-L., Sun, H.: Playful design elements and stages of player experience in gamification. In: Proceedings of the 38th International Conference on Information Systems, Seoul (2017)

    Google Scholar 

  42. Nass, C., Moon, Y.: Machines and mindlessness: social responses to computers. J. Soc. Issues 56(1), 81–103 (2000)

    Article  Google Scholar 

  43. Meuser, M., Nagel, U.: ExpertInneninterviews - vielfach erprobt, wenig bedacht. VS Verlag für Sozialwissenschaften, Wiesbaden (2002). https://doi.org/10.1007/978-3-322-93270-9_3

  44. Chandra, L., Gregor, S., Seidel, S.: Prescriptive knowledge in IS research. Conceptualizing design principles in terms of materiality, action, and boundary conditions. In: Proceedings of the 48th Hawaii International Conference on System Sciences, Kauai, Hawaii, USA (2015)

    Google Scholar 

  45. Venable, J., Pries-Heje, J., Baskerville, R.: A comprehensive framework for evaluation in design science research. In: Peffers, K., Rothenberger, M., Kuechler, B. (eds.) DESRIST 2012. LNCS, vol. 7286, pp. 423–438. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29863-9_31

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universität Hamburg, 20146, Hamburg, Germany

    Christina Wiethof, Tim Roocks & Eva A. C. Bittner

Authors
  1. Christina Wiethof
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tim Roocks
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eva A. C. Bittner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christina Wiethof .

Editor information

Editors and Affiliations

  1. Siemens (United States), Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wiethof, C., Roocks, T., Bittner, E.A.C. (2022). Gamifying the Human-in-the-Loop: Toward Increased Motivation for Training AI in Customer Service. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2022. Lecture Notes in Computer Science(), vol 13336. Springer, Cham. https://doi.org/10.1007/978-3-031-05643-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05643-7_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05642-0

  • Online ISBN: 978-3-031-05643-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation