Abstract
With the advance of technology, computer-based tools are increasingly used for writing instruction. However, there is a gap between their practical application and research on how they are used and their effects on writing skills. In the present study, we examined the use of Writing Mentor® (WM), a free Google Docs add-on designed to support academic writing through automated feedback. We used event logs to explore the activities that users-in-the-wild engaged in while revising their submissions. We found that the quality of users’ written products significantly improved from the first submission to the last. Viewing feedback related to the writing being well-edited more frequently and more time spent in WM were significantly associated with a bigger improvement in writing quality. Our findings have implications for the development of writing feedback and the design of AI-assisted tools to support writing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Burstein, J., et al.: Writing mentor: writing progress using self-regulated writing support. J. Writ. Anal. 2, 285–313 (2018). https://doi.org/10.37514/jwa-j.2018.2.1.12
Foltz, P.W., Rosenstein, M.: Data mining large-scale formative writing. In: Lang, C., Siemens, G., Wise, A., Gašević, D. (eds.) Handbook of Learning Analytics, pp. 199–210. Society for Learning Analytics Research (2017)
Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78, 153–189 (2008). https://doi.org/10.3102/0034654307313795
Wang, E.L., et al.: eRevis(ing): students’ revision of text evidence use in an automated writing evaluation system. Assess. Writ. 44, 100449 (2020). https://doi.org/10.1016/j.asw.2020.100449
Kluger, A.N., DeNisi, A.: The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol. Bull. 119, 254–284 (1996). https://doi.org/10.1037/0033-2909.119.2.254
Nunes, A., Cordeiro, C., Limpo, T., Castro, S.L.: Effectiveness of automated writing evaluation systems in school settings: a systematic review of studies from 2000 to 2020 (2022)
Smarter Balanced Assessment Consortium: Smarter Balanced Performance Task Scoring Rubrics Grades 3–11 (2014)
Attali, Y., Burstein, J.: Automated essay scoring with e-rater v.2. J. Technol. Learn. Assess. 4, 3–30 (2006)
Beigman Klebanov, B., Burstein, J., Harackiewicz, J.M., Priniski, S.J., Mulholland, M.: Reflective writing about the utility value of science as a tool for increasing STEM motivation and retention – can AI help scale up? Int. J. Artif. Intell. Educ. 27(4), 791–818 (2017). https://doi.org/10.1007/s40593-017-0141-4
Acknowledgments
We would like to thank Jill Burstein, Mengxiao Zhu, Sophia Chan, James V. Bruno, Eowyn Winchester, Hillary Molloy, Josh Crandall, Lisa Bergman, Nitin Madnani, and Martin Chodorow for their contributions to the project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, Y., Beigman Klebanov, B., Livne, O.E., Hao, J. (2023). Analyzing Users’ Interaction with Writing Feedback and Their Effects on Writing Performance. In: Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., Santos, O.C. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2023. Communications in Computer and Information Science, vol 1831. Springer, Cham. https://doi.org/10.1007/978-3-031-36336-8_72
Download citation
DOI: https://doi.org/10.1007/978-3-031-36336-8_72
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36335-1
Online ISBN: 978-3-031-36336-8
eBook Packages: Computer ScienceComputer Science (R0)