Skip to main content

Advertisement

Log in

Artificial Intelligence (AI) Student Assistants in the Classroom: Designing Chatbots to Support Student Success

  • Published:
Information Systems Frontiers Aims and scope Submit manuscript

Abstract

In higher education, low teacher-student ratios can make it difficult for students to receive immediate and interactive help. Chatbots, increasingly used in various scenarios such as customer service, work productivity, and healthcare, might be one way of helping instructors better meet student needs. However, few empirical studies in the field of Information Systems (IS) have investigated pedagogical chatbot efficacy in higher education and fewer still discuss their potential challenges and drawbacks. In this research we address this gap in the IS literature by exploring the opportunities, challenges, efficacy, and ethical concerns of using chatbots as pedagogical tools in business education. In this two study project, we conducted a chatbot-guided interview with 215 undergraduate students to understand student attitudes regarding the potential benefits and challenges of using chatbots as intelligent student assistants. Our findings revealed the potential for chatbots to help students learn basic content in a responsive, interactive, and confidential way. Findings also provided insights into student learning needs which we then used to design and develop a new, experimental chatbot assistant to teach basic AI concepts to 195 students. Results of this second study suggest chatbots can be engaging and responsive conversational learning tools for teaching basic concepts and for providing educational resources. Herein, we provide the results of both studies and discuss possible promising opportunities and ethical implications of using chatbots to support inclusive learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Asatiani, A., Malo, P., Rådberg Nagbøl, P. R., Penttinen, E., Rinta-Kahila, T., & Salovaara, A. (2021). Sociotechnical Envelopment of Artificial Intelligence: An Approach to Organizational Deployment of Inscrutable Artificial Intelligence Systems. Journal of the Association for Information Systems, 22(2), 325–352. https://doi.org/10.17705/1jais.00491

    Article  Google Scholar 

  • Ahmad, R., Siemon, D., Gnewuch, U., & Robra-Bissantz, S. (2022). Designing Personality-Adaptive Conversational Agents for Mental Health Care. Information Systems Frontiers, 1–21. https://doi.org/10.1007/s10796-022-10254-9

  • Abras, C., Maloney-Krichmar, D., & Preece, J. (2004). User-centered design. Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications, 37(4), 445–456.

  • Atif, Y. (2013). Conversational learning integration in technology enhanced classrooms. Computers in Human Behavior, 29(2), 416–423. https://doi.org/10.1016/j.chb.2012.07.026

    Article  Google Scholar 

  • Atkins, S., Badrie, I., & van Otterloo, S. (2021). Applying Ethical AI Frameworks in practice: Evaluating conversational AI chatbot solutions, Computers and Society Research Journal, 1. Retrieved May 24, 2022 from: https://ictinstitute.nl/wp-content/uploads/2021/09/Measuring_the_Responsible_Use_of_Conversational_AI__The_Application_of_AI_Frameworks_on_Chatbots-1.pdf

  • Avula, S., Chadwick, G., Arguello, J., & Capra, R. (2018). SearchBots: User Engagement with ChatBots during Collaborative Search. Proceedings of the 2018 Conference on Human Information Interaction & Retrieval, 52–61. https://doi.org/10.1145/3176349.3176380

  • Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82–115. https://doi.org/10.1016/j.inffus.2019.12.012

    Article  Google Scholar 

  • Behera, R. K., Bala, P. K., & Ray, A. (2021). Cognitive Chatbot for personalised contextual customer service: Behind the scene and beyond the hype. Information Systems Frontiers, 1–21. https://doi.org/10.1007/s10796-021-10168-y

  • Bii, P. (2013). Chatbot technology: A possible means of unlocking student potential to learn how to learn. Educational Research, 4, 218–221.

  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

    Article  Google Scholar 

  • Biran, O., & Cotton, C. (2017). Explanation and Justification in Machine Learning: A Survey, International Joint Conference on Artificial Intelligence (I JCAI-17) Workshop on Explainable AI (XAI) .

  • Carman, M., & Rosman, B. (2020). Applying a principle of explicability to AI research in Africa: Should we do it? Ethics and Information Technology, 3, 107–117. https://doi.org/10.1007/s10676-020-09534-2

    Article  Google Scholar 

  • Casey, B., Farhangi, A., & Vogl, R. (2019). Rethinking Explainable Machines: The GDPR’s “Right to Explanation” Debate and the Rise of Algorithmic Audits in Enterprise, 34. Berkeley Technology Law Journal, 34(143), 143–188.

    Google Scholar 

  • Cheng, L. (2018). Microsoft introduces guidelines for developing responsible conversational AI. Microsoft Blog. Retrieved on May 24, 2022 from: https://blogs.microsoft.com/blog/2018/11/14/microsoft-introduces-guidelines-for-developing-responsible-conversational-ai

  • Chocarro, R., Cortiñas, M., & Marcos-Matás, G. (2021). Teachers’ attitudes towards chatbots in education: a technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristics. Educational Studies. Routledge, 0(0), pp. 1–19. https://doi.org/10.1080/03055698.2020.1850426

  • Chuah, K. M., & Kabilan, M. K. (2021). Teachers' Views on the Use of Chatbots to Support English Language Teaching in a Mobile Environment. International Journal of Emerging Technologies in Learning, 16(20).

  • Cowls, J., Tsamados, A., Taddeo, M., & Floridi, L. (2021). A definition, benchmark and database of AI for social good initiatives. Nature Machine Intelligence, 3(2), 111–115. https://doi.org/10.1038/s42256-021-00296-0

    Article  Google Scholar 

  • Dobransky, N., & Frymier, A. (2004). Developing teacher–student relationships through out of class communication. Communication Quarterly, 52(3), 211–223.

    Article  Google Scholar 

  • D’Silva, G., Jani, M., Jadhav, V., Bhoir, A., & Amin, P. (2020). Career Counselling Chatbot Using Cognitive Science and Artificial Intelligence. In H. Vasudevan, A. Michalas, N. Shekokar & M. Narvekar (Eds.), Advanced Computing Technologies and Applications (pp. 1–9). Springer Singapore. https://doi.org/10.1007/978-981-15-3242-9_1

  • Eicher, B., Polepeddi, L., & Goel, A. (2018). Jill Watson Doesn’t Care if You’re Pregnant: Grounding AI Ethics in Empirical Studies. Proceedings of the 2018 AAAI/ACM Conference on Ai, Ethics, and Society, 88–94. https://doi.org/10.1145/3278721.3278760

  • Fadhil, A., & Gabrielli, S. (2017). Addressing Challenges in Promoting Healthy Lifestyles: The AI-Chatbot Approach. Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare - PervasiveHealth ’17, 261–265. https://doi.org/10.1145/3154862.3154914

  • Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017). Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Mental Health, 4(2), e19–e19. https://doi.org/10.2196/mental.7785

    Article  Google Scholar 

  • Følstad, A., Brandtzaeg, P. B., Feltwell, T., Law, E., Tscheligi, M., & Luger, E. (2018). SIG: Chatbots for Social Good. Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, 1–4. https://doi.org/10.1145/3170427.3185372

  • Ghallab, M. (2019). Responsible AI: Requirements and challenges. AI Perspectives, 1(1), 1–7. https://doi.org/10.1186/s42467-019-0003-z

    Article  Google Scholar 

  • Goel, A. K., & Polepeddi, L. (2016). Jill Watson: A Virtual Teaching Assistant for Online Education, Georgia Institute of Technology. Retrieved May 24, 2022 from https://smartech.gatech.edu/handle/1853/59104.

  • Goda, Y., Yamada, M., Matsukawa, H., Hata, K., & Yasunami, S. (2014). Conversation with a Chatbot before an Online EFL Group Discussion and the Effects on Critical Thinking. The Journal of Information and Systems in Education, 13(1), 1–7. https://doi.org/10.12937/ejsise.13.1

    Article  Google Scholar 

  • Gonda, D., Lei, C.-U., Chai, Y., Hou, X., & Tam, V. (2018). Pre-Conference Workshop-Chatbot Tutors for Blended Learning: Why Bother? And Where to Start? 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), 1211–1212. https://doi.org/10.1109/TALE.2018.8615180

  • Hagendorff, T. (2020). The Ethics of AI Ethics: An Evaluation of Guidelines. Minds and Machines (Dordrecht), 30(1), 99–120. https://doi.org/10.1007/s11023-020-09517-8

    Article  Google Scholar 

  • Han, X., Zhou, M., Turner, M. J., & Yeh, T. (2021, May). Designing Effective Interview Chatbots: Automatic Chatbot Profiling and Design Suggestion Generation for Chatbot Debugging. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1–15).

  • Heo, J., & Lee, J. (2019). CiSA: An Inclusive Chatbot Service for International Students and Academics. HCI International 2019 – Late Breaking Papers, 153–167. https://doi.org/10.1007/978-3-030-30033-3_12

  • Hobert, S. (2019). Say Hello to ’Coding Tutor’! Design and Evaluation of a Chatbot-Based Learning System Supporting Students to Learn to Program. 40th International Conference on Information Systems (ICIS), Munich, Germany.

  • Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257.

    Article  Google Scholar 

  • Hwang, G. J., & Chang, C. Y. (2021). A review of opportunities and challenges of chatbots in education. Interactive Learning Environments, 1–14. https://doi.org/10.1080/10494820.2021.1952615

  • Jain, M., Kumar, P., Bhansali, I., Liao, Q., Truong, K., & Patel, S. (2018). FarmChat: A Conversational Agent to Answer Farmer Queries. Proceedings of ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(4), 1–22. https://doi.org/10.1145/3287048

    Article  Google Scholar 

  • Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389–399. https://doi.org/10.1038/s42256-019-0088-2

    Article  Google Scholar 

  • Johannsen F. & Leist, S. (2018) Comparison of Commercial Chatbot Solutions for Supporting Customer Interaction.” European Conference on Information Systems 2018.

  • Kerly, A., Hall, P., & Bull, S. (2007). Bringing chatbots into education: Towards natural language negotiation of open learner models. Knowledge-Based Systems, 20(2), 177–185. https://doi.org/10.1016/j.knosys.2006.11.014

    Article  Google Scholar 

  • Kerry A., Ellis R., Bull S. (2009) Conversational Agents in E-Learning. In: Allen T., Ellis R., Petridis M. (Eds.), Applications and Innovations in Intelligent Systems XVI. SGAI 2008. Springer, London. https://doi.org/10.1007/978-1-84882-215-3_13

  • Kohnke, L. (2022). A Pedagogical Chatbot: A Supplemental Language Learning Tool. RELC Journal, 00336882211067054. https://doi.org/10.1177/00336882211067054

  • Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212–218.

    Article  Google Scholar 

  • Kushwaha, A. K., & Kar, A. K. (2021). MarkBot–a language model-driven chatbot for interactive marketing in post-modern world. Information Systems Frontiers, 1–18. https://doi.org/10.1007/s10796-021-10184-y

  • Lee, M., Ackermans, S., van As, N., Chang, H., Lucas, E., & IJsselsteijn, W. (2019). Caring for Vincent: A Chatbot for Self-Compassion. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–13. https://doi.org/10.1145/3290605.3300932

  • Li, J., Zhou, M. X., Yang, H., & Mark, G. (2017, March). Confiding in and listening to virtual agents: The effect of personality. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (pp. 275–286).

  • Luger, E., & Sellen, A. (2016). ‘Like Having a Really Bad PA’: The Gulf between User Expectation and Experience of Conversational Agents. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 5286–5297. https://doi.org/10.1145/2858036.2858288

  • Malik, R., Shrama, A., Trivedi, S., & Mishra, R. (2021). Adoption of Chatbots for Learning among University Students: Role of Perceived Convenience and Enhanced Performance. International Journal of Emerging Technologies in Learning (iJET), 16(18), 200–212.

    Article  Google Scholar 

  • Microsoft (2019). Artificial Intelligence for Africa: An Opportunity for Growth, Development, and Democratisation. Retrieved May 24, 2022, from https://info.microsoft.com/ME-DIGTRNS-WBNR-FY19-11Nov-02-AIinAfrica-MGC0003244_01Registration-ForminBody.html

  • Nguyen, T. H., Waizenegger, L., & Techatassanasoontorn, A. A. (2021). “Don’t Neglect the User!”–Identifying Types of Human-Chatbot Interactions and their Associated Characteristics. Information Systems Frontiers, 1–42. https://doi.org/10.1007/s10796-021-10212-x

  • Nirala, K. K., Singh, N. K., & Purani, V. S. (2022). A survey on providing customer and public administration based services using AI: chatbot. Multimedia Tools and Applications, 1–32. https://doi.org/10.1007/s11042-021-11458-y

  • Norman, D. A., & Draper, S. W. (1986). User centered system design: New perspectives on human-computer interaction.

  • Okonkwo, C. W., & Ade-Ibijola, A. (2020). Python-Bot: A Chatbot for Teaching Python Programming. Engineering Letters, 29(1).

  • Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033.

    Google Scholar 

  • Parab, A., Palkar, S., Maurya, S., & Balpande, S. (2017). An Intelligent Career Counselling Bot. International Research Journal of Engineering and Technology (IRJET), 4(3), 2325–2330.

    Google Scholar 

  • Pereira, J. (2016). Leveraging Chatbots to Improve Self-Guided Learning through Conversational Quizzes. Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality - TEEM ’16. Association for Computing Machinery, New York, NY, USA, 911–918. https://doi.org/10.1145/3012430.3012625

  • Peters, D., Vold, K., Robinson, D., & Calvo, R. A. (2020). Responsible AI-Two Frameworks for Ethical Design Practice. IEEE Transactions on Technology and Society, 1(1), 34–47. https://doi.org/10.1109/TTS.2020.2974991

    Article  Google Scholar 

  • Peterson, D. (2016). AdmitHub launches first college chatbot with Georgia State. Retrieved May 24, 2022, from https://blog.admithub.com/admithub-launches-first-college-chatbot-with-georgia-state

  • Petrović, J., & Jovanović, M. (2021). The Role of Chatbots in Foreign Language Learning: The Present Situation and the Future Outlook. In Artificial Intelligence: Theory and Applications (pp. 313–330). Springer, Cham.

  • Porra, J., Lacity, M., & Parks, M. S. (2020). Can computer based human-likeness endanger humanness?”–A philosophical and ethical perspective on digital assistants expressing feelings they can’t have. Information Systems Frontiers, 22(3), 533–547.

  • Schlesinger, A., O’Hara, K., & Taylor, A. (2018). Let’s Talk About Race: Identity, Chatbots, and AI. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–14. https://doi.org/10.1145/3173574.3173889

  • Schroeder, J., Wilkes, C., Rowan, K., Toledo, A., Paradiso, A., Czerwinski, M., Mark, G., & Linehan, M. (2018). Pocket Skills: A Conversational Mobile Web App To Support Dialectical Behavioral Therapy. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–15. https://doi.org/10.1145/3173574.3173972.

  • Stieglitz, S., Mirbabaie, M., Möllmann, N. R., & Rzyski, J. (2021). Collaborating with Virtual Assistants in Organizations: Analyzing Social Loafing Tendencies and Responsibility Attribution. Information Systems Frontiers, 1–26. https://doi.org/10.1007/s10796-021-10201-0

  • Tallyn, E., Fried, H., Gianni, R., Isard, A., & Speed, C. (2018). The Ethnobot: Gathering Ethnographies in the Age of IoT. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–13. https://doi.org/10.1145/3173574.3174178

  • Te Pas, M. E., Rutten, W. G., Bouwman, R. A., & Buise, M. P. (2020). User experience of a chatbot questionnaire versus a regular computer questionnaire: Prospective comparative study. JMIR Medical Informatics, 8(12), e21982.

    Article  Google Scholar 

  • Tomašev, N., Cornebise, J., Hutter, F., Mohamed, S., Picciariello, A., Connelly, B., Belgrave, D. C., Ezer, D., van der Haert, F. C., Mugisha, F., & Abila, G. (2020). AI for social good: Unlocking the opportunity for positive impact. Nature Communications, 11(1), 1–6.

    Article  Google Scholar 

  • Wambsganss, T., Soellner, M., & Leimeister, J.M. (2020). Design and Evaluation of an Adaptive Dialog-Based Tutoring System for Argumentation Skills. International Conference on Information Systems (ICIS).

  • Wearn, O. R., Freeman, R., & Jacoby, D. M. P. (2019). Responsible AI for conservation. Nature Machine Intelligence, 1(2), 72–73. https://doi.org/10.1038/s42256-019-0022-7

    Article  Google Scholar 

  • Weller A. (2019) Transparency: Motivations and Challenges. In: Samek W., Montavon G., Vedaldi A., Hansen L., Müller KR. (eds) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Lecture Notes in Computer Science, vol 11700. Springer, Cham. https://doi.org/10.1007/978-3-030-28954-6_2

  • Williams, A., Kaur, H., Mark, G., Thompson, A., Iqbal, S., & Teevan, J. (2018). Supporting Workplace Detachment and Reattachment with Conversational Intelligence. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–13. https://doi.org/10.1145/3173574.3173662

  • Winkler, R., & Soellner, M. (2018). Unleashing the Potential of Chatbots in Education: A State-Of-The-Art Analysis. Academy of Management Proceedings, 2018(1). https://doi.org/10.5465/AMBPP.2018.15903abstract

  • Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021). Are We There Yet? - A Systematic Literature Review on Chatbots in Education. Frontiers in Artificial Intelligence, 4, 654924. https://doi.org/10.3389/frai.2021.654924

    Article  Google Scholar 

  • Wong-Villacres, M., Evans, H., Schechter, D., DiSalvo, B., & Kumar, N. (2019). Consejero automatico: chatbots for supporting Latino parents’ educational engagement. Proceedings of the Tenth International Conference on Information and Communication Technologies and Development, 1–5. https://doi.org/10.1145/3287098.3287149

  • Xiao, Z., Zhou, M., & Fu, W.-T. (2019). Who should be my teammates: using a conversational agent to understand individuals and help teaming. Proceedings of the 24th International Conference on Intelligent User Interfaces, 437–447. https://doi.org/10.1145/3301275.3302264

  • Xiao, Z., Zhou, M. X., Chen, W., Yang, H., & Chi, C. (2020a, April). If I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skills. In Proceedings of the 2020a CHI Conference on Human Factors in Computing Systems (pp. 1–14).

  • Xiao, Z., Zhou, M. X., Liao, Q. V., Mark, G., Chi, C., Chen, W., & Yang, H. (2020b). Tell me about yourself: Using an AI-powered chatbot to conduct conversational surveys with open-ended questions. ACM Transactions on Computer-Human Interaction (TOCHI), 27(3), 1–37.

    Article  Google Scholar 

  • Xu, A., Liu, Z., Guo, Y., Sinha, V., & Akkiraju, R. (2017). A New Chatbot for Customer Service on Social Media. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17, 3506–3510.

  • Zhao, J., Song, T., & Sun, Y. (2020). Apihelper: Helping junior android programmers learn api usage. IAENG International Journal of Computer Science, 47(1), 92–97.

    Google Scholar 

  • Zhou, M., Mark, G., Li, J., & Yang, H. (2019). Trusting Virtual Agents: The Effect of Personality. ACM Transactions on Interactive Intelligent Systems, 9(2–3), 1–36. https://doi.org/10.1145/3232077

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yu Chen.

Ethics declarations

Conflict of Interest

The authors have no conflict of interests to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1: Chatbot Interview questions in Study 1

  1. 1.

    I know we just met, but could you specify your gender? If you have a preferred gender identity, please let me know.

  2. 2.

    So, tell me a bit about yourself. What is your major and do you currently work somewhere?

  3. 3.

    Based on that, how many hours a week to honestly get to study including time to do any assignments?

  4. 4.

    Nice! So, what are some questions/concerns do you frequently have about this course?

  5. 5.

    Alright, I can see how some students could potentially have questions/concerns such as those. Let's say if I were to be always available to answer your questions about Business Systems, what would be some of the main questions you hope I can help you with?

  6. 6.

    Fasho, I can handle that! So, what kind of course materials do you think I could potentially be capable of presenting to you through chatting like this?

  7. 7.

    Hmm, interesting, I think there could be potentially various ways of presenting things to people since everyone learns in different ways. Do you think I could potentially help you in a few things such as developing critical thinking skills and understanding the course material better so you can get a better grade?

  8. 8.

    Okay makes sense, what do you think is my best strength as a potential course assistant for you?

  9. 9.

    Haha I'll do my best to live up to that standard, so based on that then what would you say is my greatest weakness as a course assistant specifically to you? Because I would love to know what I could do my best to improve on!

  10. 10.

    Thank you for sharing all these wonderful examples and experiences! Do you have any questions or comments for me before we wrap this up?

Appendix 2: Chatbot interview questions in Study 2

figure afigure afigure a

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Jensen, S., Albert, L.J. et al. Artificial Intelligence (AI) Student Assistants in the Classroom: Designing Chatbots to Support Student Success. Inf Syst Front 25, 161–182 (2023). https://doi.org/10.1007/s10796-022-10291-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10796-022-10291-4

Keywords

Navigation