Skip to main content
SpringerLink
Log in

Towards the Automatic Generation of Pedagogical Conversational Agents from Lecture Slides

  • Conference paper
  • First Online:
Multimedia Technology and Enhanced Learning (ICMTEL 2021)

Abstract

Although corresponding technological and didactical models have been known for decades, the digitization of teaching has hardly advanced beyond simple non-interactive formats (e.g. downloadable slides are provided within a learning management system). The COVID-19 crisis is changing this situation dramatically, creating a high demand for highly interactive formats and fostering exchange between conversation partners about the course content. Systems are required that are able to communicate with students verbally, to answer their questions, and to check the students’ knowledge. While technological advances have made such systems possible in principle, the game stopper is the large amount of manual work and knowledge that must be put into designing such a system and feeding it the right content.

In this publication, we present a first system to overcome the aforementioned drawback by automatically generating a corresponding dialog system from slide-based presentations, such as PowerPoint, OpenOffice, or Keynote, which can be dynamically adapted to the respective students and their needs. Our first experiments confirm the proof of concept and reveal that such a system can be very handy for both respective groups, learners and lecturers, alike. The limitations of the developed system, however, also reminds us that many challenges need to be addressed to improve the feasibility and quality of such systems, in particular in the understanding of semantic knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Amazon: Alexa in higher education (2020). https://aws.amazon.com/de/education/alexa-edu/higher-education/

  2. Atapattu, T., Falkner, K., Falkner, N.: A comprehensive text analysis of lecture slides to generate concept maps. Comput. Educ. 115, 96–113 (2017)

    Article  Google Scholar 

  3. Bocklisch, T., Faulkner, J., Pawlowski, N., Nichol, A.: Rasa: open source language understanding and dialogue management. arXiv preprint arXiv:1712.05181 (2017)

  4. Cer, D.M., et al.: Universal sentence encoder. arXiv abs/1803.11175 (2018)

    Google Scholar 

  5. Ch, D.R., Saha, S.K.: Automatic multiple choice question generation from text: a survey. IEEE Trans. Learn. Technol. 13(1), 14–25 (2020). https://doi.org/10.1109/TLT.2018.2889100

  6. Damonte, M., Goel, R., Chung, T.: Practical semantic parsing for spoken language understanding. arXiv preprint arXiv:1903.04521 (2019)

  7. D’Mello, S.K., Dowell, N., Graesser, A.: Does it really matter whether students’ contributions are spoken versus typed in an intelligent tutoring system with natural language? J. Exp. Psychol. Appl. 17(1), 1 (2011)

    Article  Google Scholar 

  8. Fiorella, L., Stull, A.T., Kuhlmann, S., Mayer, R.E.: Instructor presence in video lectures: the role of dynamic drawings, eye contact, and instructor visibility. J. Educ. Psychol. 111(7), 1162 (2019)

    Article  Google Scholar 

  9. Følstad, A., Skjuve, M., Brandtzaeg, P.B.: Different chatbots for different purposes: towards a typology of chatbots to understand interaction design. In: Bodrunova, S., et al. (eds) INSCI 2018. LNCS, vol. 11551, pp. 145–156. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17705-8_13

  10. Fügen, C., et al.: Advances in lecture recognition: the ISL RT-06s evaluation system. In: Ninth International Conference on Spoken Language Processing (2006)

    Google Scholar 

  11. Gašević, D., Dawson, S., Rogers, T., Gasevic, D.: Learning analytics should not promote one size fits all: the effects of instructional conditions in predicting academic success. Internet High. Educ. 28, 68–84 (2016)

    Google Scholar 

  12. Graesser, A.C.: Conversations with autotutor help students learn. Int. J. Artif. Intell. Educ. 26(1), 124–132 (2016)

    Article  Google Scholar 

  13. Hobert, S., Berens, F.: Small talk conversations and the long-term use of chatbots in educational settings – experiences from a field study. In: Følstad, A., et al. (eds.) CONVERSATIONS 2019. LNCS, vol. 11970, pp. 260–272. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-39540-7_18

  14. Hobert, S., Meyer von Wolff, R.: Say hello to your new automated tutor–a structured literature review on pedagogical conversational agents (2019)

    Google Scholar 

  15. Jacob, L., Lachner, A., Scheiter, K.: Learning by explaining orally or in written form? Text complexity matters. Learn. Instr. 68, 101344 (2020)

    Google Scholar 

  16. Kelsey, E., Ray, F., Brown, D., Robson, R.: Design of a domain-independent, interactive, dialogue-based tutor for use within the GIFT framework. In: Generalized Intelligent Framework for Tutoring (GIFT) Users Symposium (Giftsym3), pp. 161–168 (2015)

    Google Scholar 

  17. Kim, K., Boelling, L., Haesler, S., Bailenson, J., Bruder, G., Welch, G.F.: Does a digital assistant need a body? The influence of visual embodiment and social behavior on the perception of intelligent virtual agents in ar. In: 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 105–114. IEEE (2018)

    Google Scholar 

  18. Kolss, M., Wolfel, M., Kraft, F., Niehues, J., Paulik, M., Waibel, A.: Simultaneous german-English lecture translation. In: International Workshop on Spoken Language Translation (IWSLT) 2008 (2008)

    Google Scholar 

  19. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360 (2016)

  20. Monz, C.: Machine learning for query formulation in question answering. Nat. Lang. Eng. 17(4), 425–454 (2011)

    Article  Google Scholar 

  21. Pearl, C.: Designing Voice user Interfaces: Principles of Conversational Experiences. O’Reilly Media, Inc., Sebastopol (2016)

    Google Scholar 

  22. Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)

  23. Rus, V., D’Mello, S., Hu, X., Graesser, A.: Recent advances in intelligent systems with conversational dialogue. AI Mag. 34, 42–54 (2013)

    Google Scholar 

  24. Schlobinski, P., Siever, T.: Sprachliche kommunikation in der digitalen welt. Eine repräsentative Umfrage, durchgeführt von forsa (1619–1021) (2018)

    Google Scholar 

  25. Seufert, S., Meier, C., Soellner, M., Rietsche, R.: A pedagogical perspective on big data and learning analytics: a conceptual model for digital learning support. Technol. Knowl. Learn. 24(4), 599–619 (2019)

    Article  Google Scholar 

  26. Wang, Y., Sumiya, K.: Semantic ranking of lecture slides based on conceptual relationship and presentational structure. Procedia Comput. Sci. 1(2), 2801–2810 (2010)

    Article  Google Scholar 

  27. Wei, C., Yu, Z., Fong, S.: How to build a chatbot: chatbot framework and its capabilities. In: Proceedings of the 2018 10th International Conference on Machine Learning and Computing, pp. 369–373 (2018)

    Google Scholar 

  28. Winkler, R., Söllner, M.: Towards empowering educators to create their own smart personal assistants. In: Proceedings of the 53rd Hawaii International Conference on System Sciences (2020)

    Google Scholar 

  29. Wölfel, M.: Robust automatic transcription of lectures. KIT Scientific Publishing (2009)

    Google Scholar 

  30. Wölfel, M., Schlippe, T., Stitz, A.: Voice driven type design. In: 2015 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), pp. 1–9. IEEE (2015)

    Google Scholar 

  31. Yang, Y., et al.: Multilingual universal sentence encoder for semantic retrieval (2019)

    Google Scholar 

  32. Zellou, G., Cohn, M.: Social and functional pressures in vocal alignment: differences for human and voice-AI interlocutors (2020)

    Google Scholar 

  33. Zhang, Y., Tuo, M., Yin, Q., Qi, L., Wang, X., Liu, T.: Keywords extraction with deep neural network model. Neurocomputing 383, 113–121 (2020)

    Article  Google Scholar 

  34. Zhang, Z., Takanobu, R., Zhu, Q., Huang, M., Zhu, X.: Recent advances and challenges in task-oriented dialog systems. Sci. China Technol. Sci., 1–17 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Business Information Systems, Karlsruhe University of Applied Sciences, Karlsruhe, Germany

    Matthias Wölfel

Authors
  1. Matthias Wölfel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthias Wölfel .

Editor information

Editors and Affiliations

  1. Hunan Normal University, Changsha, China

    Weina Fu

  2. University of Jinan, Jinan, China

    Yuan Xu

  3. University of Leicester, Leicester, UK

    Shui-Hua Wang

  4. University of Leicester, Leicester, UK

    Yudong Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wölfel, M. (2021). Towards the Automatic Generation of Pedagogical Conversational Agents from Lecture Slides. In: Fu, W., Xu, Y., Wang, SH., Zhang, Y. (eds) Multimedia Technology and Enhanced Learning. ICMTEL 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 388. Springer, Cham. https://doi.org/10.1007/978-3-030-82565-2_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-82565-2_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-82564-5

  • Online ISBN: 978-3-030-82565-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation