skip to main content
10.1145/3631700.3665231acmconferencesArticle/Chapter ViewAbstractPublication PagesumapConference Proceedingsconference-collections
short-paper
Open access

LLMs for Knowledge Modeling: NLP Approach to Constructing User Knowledge Models for Personalized Education

Published: 28 June 2024 Publication History

Abstract

This study proposes a method for developing a user knowledge model based on their past learning experiences. The focus is on analyzing academic data, particularly lesson records, to extract information about educational concepts. The ultimate goal is to construct a comprehensive profile that reflects the user’s accumulated knowledge throughout their learning journey. Two distinct methods are introduced for concept extraction: a gazetteer-based Named Entity Recognition approach and prompt engineering using ChatGPT. The effectiveness of these methods is assessed through a case study involving a graduate student at the University of Pisa. These knowledge profiles hold significant relevance in today’s educational landscape. With the prevalence of lifelong learning, individuals from diverse academic backgrounds participate in professional development courses. This diversity in past learning experiences can pose a challenge for instructors and course designers who must adapt lessons to be understandable and engaging for an audience with heterogeneous knowledge bases. The analysis of academic data offers a systematic approach to modeling each individual’s acquired knowledge. This, in turn, facilitates the personalization of learning content and pathways based on students’ unique learning experiences. The outcome is an inclusive learning environment that caters to the specific needs of each participant, thereby promoting compelling and stimulating learning experiences.

References

[1]
Xinyuan Chang, Bingxin Wang, and Bowen Hui. 2022. Towards an Automatic Approach for Assessing Program Competencies. In LAK22: 12th International Learning Analytics and Knowledge Conference (Online, USA) (LAK22). Association for Computing Machinery, New York, NY, USA, 119–129. https://doi.org/10.1145/3506860.3506875
[2]
Filippo Chiarello, Gualtiero Fantoni, Terence Hogarth, Vito Giordano, Liga Baltina, and Irene Spada. 2021. Towards ESCO 4.0 – Is the European classification of skills in line with Industry 4.0? A text mining approach. Technological Forecasting and Social Change 173 (2021), 121177. https://doi.org/10.1016/j.techfore.2021.121177
[3]
Grant Cooper. 2023. Examining Science Education in ChatGPT: An Exploratory Study of Generative Artificial Intelligence. Journal of Science Education and Technology (2023). https://doi.org/10.1007/s10956-023-10039-y
[4]
Joseph Crawford, Michael Cowling, and Kelly‐Ann Allen. 2023. Leadership Is Needed for Ethical ChatGPT: Character, Assessment, and Learning Using Artificial Intelligence (AI). Journal of University Teaching and Learning Practice (2023). https://doi.org/10.53761/1.20.3.02
[5]
Kartik Detroja, C.K. Bhensdadia, and Brijesh S. Bhatt. 2023. A survey on Relation Extraction. Intelligent Systems with Applications 19 (2023), 200244. https://doi.org/10.1016/j.iswa.2023.200244
[6]
Damian Eke. 2023. ChatGPT and the Rise of Generative AI: Threat to Academic Integrity?Journal of Responsible Technology (2023). https://doi.org/10.1016/j.jrt.2023.100060
[7]
Silvia Fareri, Gualtiero Fantoni, Filippo Chiarello, Elena Coli, and Anna Binda. 2020. Estimating Industry 4.0 impact on job profiles and skills using text mining. Computers in industry 118 (2020), 103222.
[8]
Rafael Ferreira-Mello, Máverick André, Anderson Pinheiro, Evandro Costa, and Cristobal Romero. 2019. Text mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 9, 6 (2019), e1332.
[9]
Mohanad Halaweh. 2023. ChatGPT in education: Strategies for responsible implementation. Contemporary Educational Technology 15, 2 (2023). https://doi.org/10.30935/cedtech/13036
[10]
Ridong Han, Tao Peng, Chaohao Yang, Benyou Wang, Lu Liu, and Xiang Wan. 2023. Is Information Extraction Solved by ChatGPT? An Analysis of Performance, Evaluation Criteria, Robustness and Errors. arxiv:2305.14450 [cs.CL]
[11]
Bo Li, Gexiang Fang, Yang Yang, Quansen Wang, Wei Ye, Wen Zhao, and Shikun Zhang. 2023. Evaluating ChatGPT’s Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness. arxiv:2304.11633 [cs.CL]
[12]
Kamil Malinka, Martin Peresíni, Anton Firc, Ondrej Hujnák, and Filip Janus. 2023. On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1. ACM. https://doi.org/10.1145/3587102.3588827
[13]
Partha Pratim Ray. 2023. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems 3 (2023), 121–154. https://doi.org/10.1016/j.iotcps.2023.04.003
[14]
Irene Spada, Filippo Chiarello, Simone Barandoni, Gianluca Ruggi, Antonella Martini, and Gualtiero Fantoni. 2022. Are universities ready to deliver digital skills and competences? A text mining-based case study of marketing courses in Italy. Technological Forecasting and Social Change 182 (2022), 121869.
[15]
Ying Su and Yong Zhang. 2020. Automatic Construction of Subject Knowledge Graph based on Educational Big Data. In Proceedings of the 2020 3rd International Conference on Big Data and Education (London, United Kingdom) (ICBDE ’20). Association for Computing Machinery, New York, NY, USA, 30–36. https://doi.org/10.1145/3396452.3396458
[16]
Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, Yong Jiang, and Wenjuan Han. 2023. Zero-Shot Information Extraction via Chatting with ChatGPT. arxiv:2302.10205 [cs.CL]

Cited By

View all
  • (2024)Implementation and Evaluation of LLM-Based Conversational Systems on a Low-Cost Device2024 IEEE Global Humanitarian Technology Conference (GHTC)10.1109/GHTC62424.2024.10771565(392-399)Online publication date: 23-Oct-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
UMAP Adjunct '24: Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization
June 2024
662 pages
ISBN:9798400704666
DOI:10.1145/3631700
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2024

Check for updates

Author Tags

  1. Information Extraction
  2. Large Language Model
  3. Name Entity Recognition
  4. Personalized Learning
  5. Prompt Engineering ChatGPT
  6. User Knowledge Model
  7. User Modeling

Qualifiers

  • Short-paper
  • Research
  • Refereed limited

Conference

UMAP '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 162 of 633 submissions, 26%

Upcoming Conference

UMAP '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)399
  • Downloads (Last 6 weeks)60
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Implementation and Evaluation of LLM-Based Conversational Systems on a Low-Cost Device2024 IEEE Global Humanitarian Technology Conference (GHTC)10.1109/GHTC62424.2024.10771565(392-399)Online publication date: 23-Oct-2024

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media