Abstract
Student feedback is crucial for evaluating the performance of teachers and the quality of teaching. Free-form text comments obtained from open-ended questions are seldom analyzed comprehensively since it is difficult to interpret and score compared to standardized rating scales. To solve this problem, the present study employed aspect-level sentiment analysis using deep learning and dictionary-based approaches to automatically calculate the emotion orientation of text-based feedback. The results showed that the model using the topic dictionary as input and the attention mechanism had the strongest prediction effect in student review sentiment classification, with a precision rate of 80%, a recall rate of 79% and an F1 value of 79%. The findings identified issues that were not otherwise apparent from analyses of purely quantitative data, providing a deeper and more constructive understanding of curriculum and teaching performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Available on request.
References
Alhija, F. N. A., & Fresko, B. (2009). Student evaluation of instruction: What can be learned from students’ written comments? Studies in Educational Evaluation, 35(1), 37–44. https://doi.org/10.1016/j.stueduc.2009.01.002
Annan, S. L., Tratnack, S., Rubenstein, C., Metzler-Sawin, E., & Hulton, L. (2013). An integrative review of student evaluations of teaching: Implications for evaluation of nursing faculty. Journal of Professional Nursing, 29(5), e10–e24. https://doi.org/10.1016/j.profnurs.2013.06.004
Aung, K. Z., & Myo, N. N. (2017). Sentiment analysis of students’ comment using lexicon based approach. In 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (pp. 149–154)
Beran, T. N., & Rokosh, J. L. (2009). Instructors’ perspectives on the utility of student ratings of instruction. Instructional Science, 37(2), 171–184. https://doi.org/10.1007/s11251-007-9045-2
Beran, T., Violato, C., & Kline, D. (2007). What’s the “use” of student ratings of instruction for administrators? One university’s experience. Canadian Journal of Higher Education, 37(1), 27–43
Bhatnagar, V., Goyal, M., & Hussain, M. A. (2018). A novel aspect based framework for tourism sector with improvised aspect and opinion mining algorithm. International Journal of Rough Sets and Data Analysis, 5(2), 119–130. https://doi.org/10.4018/ijrsda.2018040106
Bing, L. (2012). Sentiment Analysis and Opinion Mining (Synthesis Lectures on Human Language Technologies). Morgan & Claypool Publishers
Brockx, B., Van Roy, K., & Mortelmans, D. (2012). The student as a commentator: Students’ comments in student evaluations of teaching. Procedia-Social and Behavioral Sciences, 69, 1122–1133. https://doi.org/10.1016/j.sbspro.2012.12.042
Chauhan, G. S., Agrawal, P., & Meena, Y. K. (2018). Aspect-Based Sentiment Analysis of Students’ Feedback to Improve Teaching–Learning Process. Smart Innovation, Systems and Technologies, 259–266.https://doi.org/10.1007/978-981-13-1747-7_25
Chong, C., Sheikh, U. U., Samah, N. A., & Ahmad Zuri Sha’ameri. (2020). &. Analysis on Reflective Writing Using Natural Language Processing and Sentiment Analysis. IOP Conference Series.Materials Science and Engineering, 884(1), 1–8. https://doi.org/10.1088/1757-899X/884/1/012069
Clayson, D. E., & Haley, D. A. (2011). Are students telling us the truth? A critical look at the student evaluation of teaching. Marketing Education Review, 21(2), 101–112. https://doi.org/10.2753/mer1052-8008210201
Denson, N., Loveday, T., & Dalton, H. (2010). Student evaluation of courses: What predicts satisfaction? Higher Education Research & Development, 29, 339–356. https://doi.org/10.1080/07294360903394466
Donnon, T., Delver, H., & Beran, T. (2010). Student and teaching characteristics related to ratings of instruction in medical sciences graduate programs. Medical Teacher, 32(4), 327–332. https://doi.org/10.3109/01421590903480097
Elhassan, K. (2009). Investigating substantive and consequential validity of student ratings of instruction. Higher Education Research & Development, 28(3), 319–333. https://doi.org/10.1080/07294360902839917
Emerson, R. J., & Records, K. (2007). Design and testing of classroom and clinical teaching evaluation tools for nursing education. International Journal of Nursing Education Scholarship (IJNES), 4(1), 16. https://doi.org/10.2202/1548-923x.1375
Greenwald, A. G. (1997). Validity concerns and usefulness of student ratings of instruction. American Psychologist, 52(11), 1182–1186. https://doi.org/10.1037/0003-066x.52.11.1182
Gupta, V., Singh, V. K., Mukhija, P., & Ghose, U. (2019). Aspect-based sentiment analysis of mobile reviews. Journal of Intelligent and Fuzzy Systems, 36(5), 4721–4730. https://doi.org/10.3233/JIFS-179021
Hammond, I., Taylor, J., & McMenamin, P. (2003). Value of a structured participant evaluation questionnaire in the development of a surgical education program. Australian and New Zealand Journal of Obstetrics and Gynaecology, 43(2), 115–118. https://doi.org/10.1046/j.0004-8666.2003.00037.x
Hodges, L. C., & Stanton, K. (2007). Translating comments on student evaluations into the language of learning. Innovative Higher Education, 31(5), 279–286. https://doi.org/10.1007/s10755-006-9027-3
Hong, W., & Li, M. (2019). A review: Text sentiment analysis methods. Computer Engineering & Science, 41(4), 750–757
Hoon, A., Oliver, E., Szpakowska, K., & Newton, P. (2014). Use of the ‘stop, start, continue’ method is associated with the production of constructive qualitative feedback by students in higher education. Assessment & Evaluation in Higher Education, 755–767. https://doi.org/10.1080/02602938.2014.956282
Kulik, J. A. (2001). Student ratings: Validity, utility, and controversy. New Directions for Institutional Research, 2001(109), 9–25. https://doi.org/10.1002/ir.1
Lin, Q., Zhu, Y., Zhang, S., Shi, P., Guo, Q., & Niu, Z. (2019). Lexical based automated teaching evaluation via students’ short reviews. Computer Applications in Engineering Education, 27(1), 194–205. https://doi.org/10.1002/cae.22068
Li, W., Jin, B., & Quan, Y. (2020). Review of research on text sentiment analysis based on deep learning. Open Access Library Journal, 7, 1–8. https://doi.org/10.4236/oalib.1106174
Medhat, W., Hassan, A., & Korashy, H. (2014). Sentiment analysis algorithms and applications: A survey. Ain Shams Engineering Journal, 5(4), 1093–1113. https://doi.org/10.1016/j.asej.2014.04.011
Nasser, F., & Fresko, B. (2002). Faculty views of student evaluation of college teaching. Assessment & Evaluation in Higher Education, 27(2), 187–198. https://doi.org/10.1080/02602930220128751
Onwuegbuzie, A. J., Daniel, L. G., & Collins, K. M. (2009). A meta-validation model for assessing the score-validity of student teaching evaluations. Quality & Quantity, 43(2), 197–209. https://doi.org/10.1007/s11135-007-9112-4
Ory, J. C. (2000). Teaching evaluation: Past, present, and future. New Directions for Teaching and Learning, 83, 13–18. https://doi.org/10.1002/tl.8302
Rajput, Q., Haider, S., & Ghani, S. (2016). Lexicon-based sentiment analysis of teachers’ evaluation. Applied Computational Intelligence and Soft Computing, 1–12. https://doi.org/10.1155/2016/2385429
Serdyukova, N., Tatum, B. C., & Serdyukova, P. (2010). Student evaluations of courses and teachers.Publication of National University,173
Shaikh, S., & Doudpotta, S. M. (2019). Aspects based opinion mining for teacher and course evaluation. Sukkur IBA Journal of Computing and Mathematical Sciences, 3(1), 34–43
Sindhu, I., Daudpota, S. M., Badar, K., Bakhtyar, M., Baber, J., & Nurunnabi, M. (2019). Aspect-based opinion mining on student’s feedback for faculty teaching performance evaluation. Ieee Access : Practical Innovations, Open Solutions, 7, 108729–108741. https://doi.org/10.1109/ACCESS.2019.2928872
Smith, C. (2008). Building effectiveness in teaching through targeted evaluation and response: Connecting evaluation to teaching improvement in higher education. Assessment & Evaluation in Higher Education, 33(5), 517–533. https://doi.org/10.1080/02602930701698942
Srinvas, A., & Hanumanthappa, M. (2017). Viable modern approaches for sentiment analysis: A survey. International Journal of Advanced Research in Computer Science, 8(7), 115–120. https://doi.org/10.26483/ijarcs.v8i7.4095
Stupans, I., McGuren, T., & Babey, A. M. (2016). Student evaluation of teaching: A study exploring student rating instrument free-form text comments. Innovative Higher Education, 41(1), 33–42. https://doi.org/10.1007/s10755-015-9328-5
Sun, J. (2012). Jieba Chinese word segmentation tool. (2018-01-21)[2018-06-25]. Retrieved from https://github.com/fxsjy/jieba
Tenzin, D., Lemay, D. J., Basnet, R. B., & Bazelais, P. (2020). Predictive analytics in education: a comparison of deep learning frameworks. Education and Information Technologies, 25(3), 1951–1963. https://doi.org/10.1007/s10639-019-10068-4
Tseng, C. W., Chou, J. J., & Tsai, Y. C. (2018). Text mining analysis of teaching evaluation questionnaires for the selection of outstanding teaching faculty members. Ieee Access : Practical Innovations, Open Solutions, 6, 72870–72879. https://doi.org/10.1109/ACCESS.2018.2878478
Wang, H. D. (2018). Multi-grain sentiment analysis of teaching reviews based on topic (pp. 25–26). Guang Zhou: South China University of Technology Press
Zhang, J., Chen, F. L., & Zhang, P. Y. (2019). The role and implementation of students’ sentiment analysis in curriculum teaching evaluation. Computer Knowledge and Technology, 15(4), 184–188
Funding
This paper was supported by the National Natural Science Foundation of China (Grant No: U1911201).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that there is no conflict of interest.
Ethics approval
The participants were protected by hiding their personal information during the research process. They knew that their participation was voluntary and they could withdraw from the study at any time.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ren, P., Yang, L. & Luo, F. Automatic scoring of student feedback for teaching evaluation based on aspect-level sentiment analysis. Educ Inf Technol 28, 797–814 (2023). https://doi.org/10.1007/s10639-022-11151-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-022-11151-z