Abstract
This paper presents the design and evaluation of an automated writing evaluation system that integrates natural language processing (NLP) and user interface design to support students in an important writing skill, namely, self-monitored revising. Results from a classroom deployment suggest that NLP can accurately analyze where and what kind of revisions students make across paper drafts, that students engage in self-monitored revising, and that the interfaces for visualizing the NLP results are perceived by students to be useful.
Supported by the National Science Foundation under Grant #173572.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Afrin, T., Kashefi, O., Olshefski, C., Litman, D., Hwa, R., Godley, A.: Effective interfaces for student-driven revision sessions for argumentative writing. In: Proceedings CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2021)
Afrin, T., Litman, D.: Annotation and classification of sentence-level revision improvement. In: Proceedings 13th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 240–246, New Orleans, Louisiana, June 2018
Burstein, J., Riordan, B., McCaffrey, D.: Expanding automated writing evaluation. In: Handbook of Automated Scoring, pp. 329–346. Chapman and Hall/CRC (2020)
Crossley, S.A., Allen, L.K., McNamara, D.S.: The writing pal: a writing strategy tutor. In: Adaptive Educational Technologies for Literacy Instruction, pp. 204–224, Routledge (2016)
Deane, P., Wilson, J., Zhang, M., Li, C., van Rijn, P., Guo, H., Roth, A., Winchester, E., Richter, T.: The sensitivity of a scenario-based assessment of written argumentation to school differences in curriculum and instruction. Int. J. Artif. Intell. Educ. 31(1), 57–98 (2021)
Faigley, L., Witte, S.: Analyzing revision. Coll. Compos. Commun. 32(4), 400–414 (1981)
Foltz, P.W., Rosenstein, M.: Data mining large-scale formative writing. In: Handbook of Learning Analytics, p. 199 (2017)
Holden, H., Rada, R.: Understanding the influence of perceived usability and technology self-efficacy on teachers’ technology acceptance. J. Res. Technol. Educ. 43(4), 343–367 (2011)
Kashefi, O., et al.: Argrewrite v. 2: an annotated argumentative revisions corpus. Language Resources and Evaluation, pp. 1–35 (2022)
MacArthur, C., Philippakos, Z., Ianetta, M.: Self-regulated strategy instruction in college developmental writing. J. Educ. Psychol. 107(3), 855 (2015)
Mayfield, E., Butler, S.: Districtwide implementations outperform isolated use of automated feedback in high school writing. In: International Conference of the Learning Sciences, vol. 2128, London, UK (2019)
Roscoe, R.D., McNamara, D.S.: Writing pal: feasibility of an intelligent writing strategy tutor in the high school classroom. J. Educ. Psychol. 105(4), 1010–1025 (2013)
Roscoe, R.D., Snow, E.L., Allen, L.K., McNamara, D.S.: Automated detection of essay revising patterns: applications for intelligent feedback in a writing tutor. Technol. Instr. Cogn. Learn. 10(1), 59–79 (2015)
Shibani, A.: Constructing automated revision graphs: a novel visualization technique to study student writing. In: Bittencourt, I.I., Cukurova, M., Muldner, K., Luckin, R., Millán, E. (eds.) AIED 2020. LNCS (LNAI), vol. 12164, pp. 285–290. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52240-7_52
Wang, E.L., et al.: erevis(ing): students’ revision of text evidence use in an automated writing evaluation system. Assessing Writing 44, 100449 (2020)
Wilson, J., Huang, Y., Palermo, C., Beard, G., MacArthur, C.A.: Automated feedback and automated scoring in the elementary grades: Usage, attitudes, and associations with writing outcomes in a districtwide implementation of mi write. Int. J. Artif. Intell. Educ., 1–43 (2021)
Wingate, U.: The impact of formative feedback on the development of academic writing. Assess. Eval. High. Educ. 35(5), 519–533 (2010)
Zhang, F., Hwa, R., Litman, D., B. Hashemi, H.: Argrewrite: a web-based revision assistant for argumentative writings. In: Proceedings f NAACL Conference: Demonstrations, San Diego, California, pp. 37–41 (2016)
Zhang, F., Litman, D.: Annotation and classification of argumentative writing revisions. In: Proceedings of the 10th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 133–143. Denver, Colorado, June 2015
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Litman, D., Afrin, T., Kashefi, O., Olshefski, C., Godley, A., Hwa, R. (2022). An Automated Writing Evaluation System for Supporting Self-monitored Revising. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2022. Lecture Notes in Computer Science, vol 13355. Springer, Cham. https://doi.org/10.1007/978-3-031-11644-5_52
Download citation
DOI: https://doi.org/10.1007/978-3-031-11644-5_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11643-8
Online ISBN: 978-3-031-11644-5
eBook Packages: Computer ScienceComputer Science (R0)