Abstract
Modern educational technology systems allow learners to access large amounts of learning materials such as educational videos, learning notes, and teaching books. Automated summarization techniques simplify the access and exploration of complex data collections by producing synthetic versions of the original content. This paper addresses the problem of video lecture summarization by means of abstractive techniques. To enhance the accessibility of the video lecture content in challenging contexts or while coping with learners with special needs it produces a synthetic textual summary condensing the key concepts mentioned in the lecture’s speech. Unlike prior works based on extractive methods, the proposed method can produce more readable and actionable summaries, not necessarily composed of existing portions of speech content. To compensate the lack of annotated data, it also opportunistically reuses the pretrained models available for meeting summarization. The experimental results achieved on a benchmark dataset show that the proposed method generates more fluent and actionable summaries than prior approaches simply relying on content extraction. Finally, we also envision further applications of summarization techniques to learning content. The future prospects of use of summarization techniques in education have shown to go well beyond video summarization.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets analyzed during and/or analysed during the current study are available at https://ocw.mit.edu/
Notes
https://ocw.mit.edu/about/ Latest access: August 2022
http://uis.unesco.org/en/topic/international-standard-classification-education-isced Latest access: May 2022
https://cloud.google.com/speech-to-text Latest access: May 2022
https://pypi.org/project/fastpunct/ Latest access: May 2022
https://pypi.org/project/pytextrank/ Latest access: June 2022
https://pypi.org/project/sentence-transformers/0.3.0/ Latest access: June 2022
https://huggingface.co/docs/transformers/ Latest access: September 2022
https://ocw.mit.edu/about/ Latest access: August 2022
References
Abhilash, R. K., Anurag, C., & Avinash, V. (2021). Lecture video summarization using subtitles. In A. Haldorai, A. Ramu, & S. Mohanram (Eds.), 2nd EAI International Conference on Big Data Innovation for Sustainable Cognitive Computing (pp. 83–92). Cham: Springer International Publishing.
Alam, T., Khan, A., & Alam, F. (2020). Punctuation restoration using transformer models for high-and low-resource languages. In: Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020). Association for Computational Linguistics, Online (pp.132–142). https://doi.org/10.18653/v1/2020.wnut-1.18. https://aclanthology.org/2020.wnut-1.18
Atapattu, T., & Falkner, K. (2018). Impact of lecturer’s discourse for students’ video engagement: Video learning analytics case study of moocs. J Learn Anal, 5(3). https://doi.org/10.18608/jla.2018.53.12
Baralis, E., & Cagliero, L. (2016). Learning from summaries: Supporting e-learning activities by means of document summarization. IEEE Transactions on Emerging Topics in Computing, 4(3), 416–428. https://doi.org/10.1109/TETC.2015.2493338
Baralis, E., & Cagliero, L. (2018). Highlighter: Automatic highlighting of electronic learning documents. IEEE Trans Emerg Top Comput, 6(1), 7–19. https://doi.org/10.1109/TETC.2017.2681655
Benedetto, I., Canale, L., Farinetti, L., Cagliero, L., & Quatra, M. (2022). Leveraging summarization techniques in educational technology systems. 46th IEEE Annual Computers, Software, And Applications Conferenc, COMPSAC 2022, Los Alamitos, CA, USA, June 27 - July 1, 2022 (pp. 415-416). https://doi.org/10.1109/COMPSAC54236.2022.00068
Borsos, Z., Marinier, R., & Vincent, D. (2022). Audiolm: a language modeling approach to audio generation. https://doi.org/10.48550/arXiv.2209.03143. arXiv:2209.03143
Cagliero, L., Farinetti, L., & Baralis, E. (2019). Recommending personalized summaries of teaching materials. IEEE Access, 7:22,729–22,739. https://doi.org/10.1109/ACCESS.2019.2899655
Carletta, J., Ashby, S., & Bourban, S. (2005). The ami meeting corpus: A pre-announcement.https://doi.org/10.1007/11677482_3
Chandrasekaran, D., & Mago, V. (2021). Evolution of semantic similarity-a survey. ACM Computing Surveys, 54(2), 1–37. https://doi.org/10.1145/3440755
Choudary, C., & Liu, T. (2007). Summarization of visual content in instructional videos. IEEE Transactions on Multimedia, 9(7), 1443–1455. https://doi.org/10.1109/TMM.2007.906602
Devlin, J., Chang, M.W., & Lee, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805
El-Kassas, W. S., Salama, C. R., & Rafea, A. A. (2021). Automatic text summarization: A comprehensive survey. Expert Systems with Applications, 165(113), 679. https://doi.org/10.1016/j.eswa.2020.113679
Fujii, Y., Yamamoto, K., & Kitaoka, N. (2008) Class lecture summarization taking into account consecutiveness of important sentences. 2438–2441
Garg, S. (2017). Automatic text summarization of video lectures using subtitles. In: Patnaik, S., & Popentiu-Vladicescu, F. (Eds.) Recent Developments in Intelligent Computing, Communication and Devices (pp. 45–52). Springer Singapore, Singapore
Gliwa, B., Mochol, I., & Biesek, M. (2019) Samsum corpus: A human-annotated dialogue dataset for abstractive summarization. arXiv:1911.12237
Gottipati, S., Shankararaman, V., & Ramesh, R. (2019). TopicSummary: A Tool for Analyzing Class Discussion Forums using Topic Based Summarizations. In: 2019 IEEE Frontiers in Education Conference (FIE) (pp. 1–9). IEEE, Covington, KY, USA. https://doi.org/10.1109/FIE43999.2019.9028526. https://ieeexplore.ieee.org/document/9028526/
Goularte, F. B., Nassar, S. M., & Fileto, R. (2019). A text summarization method based on fuzzy rules and applicable to automated assessment. Expert Systems with Applications, 115, 264–275. https://doi.org/10.1016/j.eswa.2018.07.047, . https://www.linkinghub.elsevier.com/retrieve/pii/S0957417418304743
Hermann, K.M., Kocisky, T., & Grefenstette, E. (2015). Teaching machines to read and comprehend. In: NIPS
Janin, A., Baron, D., & Edwards, J. (2003). The icsi meeting corpus. I–364. https://doi.org/10.1109/ICASSP.2003.1198793
Khalil, M., Prinsloo, P., & Slade, S. (2022). A comparison of learning analytics frameworks: A systematic review. In: LAK22: 12th International Learning Analytics and Knowledge Conference (pp. 152-163). Association for Computing Machinery, New York, NY, USA, LAK22. https://doi.org/10.1145/3506860.3506878
Lee, H., Liu, M., & Riaz, H. (2021). Attention based video summaries of live online zoom classes. https://dblp.org/rec/journals/corr/abs-2101-06328.bib
Lewis, M., Liu, Y., & Goyal, N. (2019) Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. https://aclanthology.org/2020.acl-main.703/
Lin, C.Y. (2004). Rouge: A package for automatic evaluation of summaries. p 10
Litvak, M., & Vanetik, N. (2017). Query-based summarization using MDL principle. In: Proceedings of the MultiLing 2017 Workshop on Summarization and Summary Evaluation Across Source Types and Genres (pp. 22–31). Association for Computational Linguistics, Valencia, Spain. https://doi.org/10.18653/v1/W17-1004. https://aclanthology.org/W17-1004
Lv, T., Cui, L., & Vasilijevic, M. (2021). Vt-ssum: A benchmark dataset for video transcript segmentation and summarization. arXiv:2106.05606
Mihalcea, R., & Tarau, P. (2004). TextRank: Bringing order into text. In:Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (pp. 404–411). Association for Computational Linguistics, Barcelona, Spain. https://aclanthology.org/W04-3252
Miller, D. (2019). Leveraging BERT for extractive text summarization on lectures. arXiv:1906.04165
Mitchell, A., Petter, S., & Harris, A. (2017). Learning by doing: Twenty successful active learning exercises for information systems courses. Journal of Information Technology Education : Innovations in Practice, 16:21–46. https://doi.org/10.28945/3643
Page, L., Brin, S., & Motwani, R. (1999). The pagerank citation ranking: Bringing order to the web. Stanford InfoLab: Tech. rep.
Parmanto, B., Ferrydiansyah, R., & Saptono, A. (2005) Access: Accessibility through simplification and summarization. In: Proceedings of the 2005 International Cross-Disciplinary Workshop on Web Accessibility (W4A) (pp. 18–25). Association for Computing Machinery, New York, NY, USA, W4A ’05. https://doi.org/10.1145/1061811.1061815
Pedrotti, M., & Nistor, N. (2014). Online lecture videos in higher education: Acceptance and motivation effects on students’ system use. In: IEEE 14th International Conference on Advanced Learning Technologies (pp. 477–479). ICALT 2014, Athens, Greece, July 7-10, 2014. IEEE Computer Society. https://doi.org/10.1109/ICALT.2014.141
Pi, Z., Zhang, Y., & Xu, K. (2022). Does an outline of contents promote learning from videos? a study on learning performance and engagement. Education and Information Technologies, 28, 3493–3511. https://doi.org/10.1007/s10639-022-11361-5
Pramudianto, F., Chhabra, T., & Gehringer, E. (2016). Assessing the quality of automatic summarization for peer review in education. In: EDM
Rahman, M. R., Shah, S., & Subhlok, J. (2020). Visual summarization of lecture video segments for enhanced navigation.https://doi.org/10.1109/ISM.2020.00033
Romero, C., & Ventura, S. (2020). Educational data mining and learning analytics: An updated survey. WIREs Data Mining and Knowledge Discovery 10(3):e1355. https://doi.org/10.1002/widm.1355. https://onlinelibrary.wiley.com/doi/abs/10.1002/widm.1355. arXiv:10.1002/widm.1355
Saini, M., Arora, V., & Singh, M. (2022). Artificial intelligence inspired multilanguage framework for note-taking and qualitative content-based analysis of lectures. Education and Information Technologies, 1–23
Shimada, A., Okubo, F., & Yin, C. (2018). Automatic summarization of lecture slides for enhanced student preview-technical report and user study. IEEE Transactions on Learning Technologies 11(2):165–178. https://doi.org/10.1109/TLT.2017.2682086, funding Information: This research was partially supported by ”PRESTO”, Japan Science and Technology Agency (JST) Japan, and ”Research and Development on Fundamental and Utilization Technologies for Social Big Data” (178A03), the Commissioned Research of the National Institute of Information and Communications Technology (NICT) Japan. Publisher Copyright: 2008-2011 IEEE.
Tan, B., Qin, L., & Xing, E.P. (2020). Summarizing text on any aspects: A knowledge-informed weakly-supervised approach. arXiv:2010.06792
Tilk, O., & Alumäe, T. (2016). Bidirectional recurrent neural network with attention mechanism for punctuation restoration. In: INTERSPEECH
Wang, F., & Chen, Z. (2018). Self-attention based network for punctuation restoration. In: 2018 24th International Conference on Pattern Recognition (ICPR), 2803–2808. https://doi.org/10.1109/ICPR.2018.8545470
Yoo, T., Jeong, H., & Lee, D. (2021). Lectys: A system for summarizing lecture videos on youtube. In: 26th International Conference on Intelligent User Interfaces - Companion (p 90–92). Association for Computing Machinery, New York, NY, USA, IUI ’21 Companion. https://doi.org/10.1145/3397482.3450722
Zhang, J., Zhao, Y., & Saleh M. (2019a). PEGASUS: pre-training with extracted gap-sentences for abstractive summarization. arXiv:1912.08777
Zhang, T., Kishore V., & Wu, F., (2019b). Bertscore: Evaluating text generation with BERT. arXiv:1904.09675
Zhu, C., Xu, R., & Zeng, M. (2020). A hierarchical network for abstractive meeting summarization with cross-domain pretraining. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. https://www.microsoft.com/en-us/research/publication/end-to-end-abstractive-summarization-for-meetings/
Zhu, J., Li, H., & Liu, T. (2018). MSMO: Multimodal summarization with multimodal output. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (pp. 4154–4164). Association for Computational Linguistics, Brussels, Belgium. https://doi.org/10.18653/v1/D18-1448. https://aclanthology.org/D18-1448
Funding
The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.
Author information
Authors and Affiliations
Contributions
All authors contributed equally to this research work. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Informed consent
The paper is an extended version of the preliminary work presented in Benedetto et al. (2022). Unlike the prior work, the current manuscript contains \(\bullet \) An overview of the existing benchmark datasets for video lecture summarization (see Section 2.1 of the current manuscript). \(\bullet \) A more thorough description of the presented methodology (see Section 2.2). \(\bullet \) A validation of the summaries generated from the open-source video lectures available in the MIT OpenCourseWare repository (see Sections 2.3, 2.4, and 2.5). \(\bullet \) A more extensive overview of the related works on summarization in education (See Section 3). \(\bullet \) A discussion of the future prospects of use of summarization techniques in education (See Section 4).
Competing interests
The authors have no relevant financial or non-financial interests to disclose.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Benedetto, I., La Quatra, M., Cagliero, L. et al. Abstractive video lecture summarization: applications and future prospects. Educ Inf Technol 29, 2951–2971 (2024). https://doi.org/10.1007/s10639-023-11855-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-023-11855-w