Skip to main content
SpringerLink
Log in

Systematic Literature Review of Automated Writing Evaluation as a Formative Learning Tool

  • Conference paper
  • First Online:
Transforming Learning with Meaningful Technologies (EC-TEL 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11722))

Included in the following conference series:

Abstract

Automated Writing Evaluation (AWE) has been increasingly used to provide writing feedback in ESL and EFL classrooms. However, research into the use of these technologies is not only scarce, but theoretically and methodologically fragmented, making it hard to draw any conclusions about their effectiveness as tools for formative evaluation. This paper reviews 29 studies into the use of AWE in ESL/EFL classrooms conducted between 2007 and 2018, analysing their theoretical and methodological underpinnings. There were two main findings. First, current AWE research ignores theoretical constructs informing other research into the use of technologies in the classroom. Second, AWE research copies the methodology used for general written corrective feedback research without using the extra tools afforded by the technology. Future AWE research should take advantage of the wealth of available theory regarding the use and implementation of technology into classrooms, as well as the new methodological possibilities offered by the technology itself.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chen, C.-F., Cheng, W.-Y.E.: Beyond the design of automated writing evaluation: pedagogical practices and perceived learning effectiveness in EFL writing classes. Lang. Learn. Technol. 12(2), 94–112 (2008)

    Google Scholar 

  2. Ranalli, J., Link, S., Chukharev-Hudilainen, E.: Automated writing evaluation for formative assessment of second language writing: investigating the accuracy and usefulness of feedback as part of argument-based validation. Educ. Psychol. 37(1), 8–25 (2017)

    Article  Google Scholar 

  3. Stevenson, M.: A critical interpretative synthesis: the integration of automated writing evaluation into classroom writing instruction. Comput. Compos. 42, 1–16 (2016)

    Article  Google Scholar 

  4. Wang, P.: Can automated writing evaluation programs help students improve their English writing? Int. J. Appl. Linguist. Engl. Lit. 2(1), 6–12 (2013)

    Article  Google Scholar 

  5. Stevenson, M., Phakiti, A.: The effects of computer-generated feedback on the quality of writing. Assessing Writ. 19, 51–65 (2014)

    Article  Google Scholar 

  6. Cotos, E.: Potential of automated writing evaluation feedback. CALICO J. 28(2), 420–459 (2011)

    Article  Google Scholar 

  7. Bai, L., Hu, G.: In the face of fallible AWE feedback: how do students respond? Educ. Psychol. 37(1), 67–81 (2017)

    Article  Google Scholar 

  8. El Ebyary, K., Windeatt, S.: The impact of computer based feedback on students’ written work. Int. J. Engl. Stud. 10(2), 121–142 (2010)

    Article  Google Scholar 

  9. Fang, Y.: Perceptions of the computer-assisted writing program among EFL college learners. Educ. Technol. Soc. 13(3), 246–256 (2010)

    Google Scholar 

  10. Lai, Y.H.: Which do students prefer to evaluate their essays: peers or computer program. Brit. J. Educ. Technol. 41(3), 432–454 (2010)

    Article  Google Scholar 

  11. Lavolette, E., Polio, C., Kahng, J.: The accuracy of computer-assisted feedback and students’ responses to it. Lang. Learn. Technol. 19(2), 50–68 (2015)

    Google Scholar 

  12. Li, S.: The effectiveness of corrective feedback in SLA: a meta-analysis. Lang. Learn. 60(2), 309–365 (2010)

    Article  Google Scholar 

  13. Liao, H.C.: Using automated writing evaluation to reduce grammar errors in writing. ELT J. 70(3), 308–319 (2016)

    Article  Google Scholar 

  14. Huang, S., Renandya, W.A.: Exploring the integration of automated feedback among lower-proficiency EFL learners. Innov. Lang. Learn. Teach. 1–12 (2018). https://www.tandfonline.com/action/showCitFormats?doi=10.1080%2F17501229.2018.1471083

  15. Xia, L., Zhong, L.: 作文自动评价系统在大学英语写作教学中的实证研究. Res. Teach. 40(1), 57–61 (2017)

    Google Scholar 

  16. Yang, X., Dai, Y.: 基于批改网的大学英语自主写作教学模式实践研究. Comput. Assist. Foreign Lang. Educ. 162, 17–23 (2015)

    Google Scholar 

  17. Zhang, Z.V.: Student engagement with computer-generated feedback: a case study. ELT J. 71(3), 317–328 (2017)

    Google Scholar 

  18. Silva, T.: Toward an understanding of the distinct nature of L2 writing: the ESL research and its implications. TESOL Q. 27(4), 657–677 (1993)

    Article  MathSciNet  Google Scholar 

  19. Truscott, J.: The case against grammar correction in L2 writing classes. Lang. Learn. 46(2), 327–369 (1996)

    Article  Google Scholar 

  20. Truscott, J.: The effect of error correction on learners’ ability to write accurately. J. Second Lang. Writ. 16, 255–272 (2007)

    Article  Google Scholar 

  21. Andrade, M., Evans, N.: Principles and Practices for Response in Second Language Writing: Developing Self-Regulated Learners. Routledge, New York (2013)

    Google Scholar 

  22. Ferris, D.R.: Written corrective feedback in second language acquisition and writing studies. Lang. Teach. 45(4), 446–459 (2012)

    Article  Google Scholar 

  23. Kang, E., Han, Z.: The efficacy of written corrective feedback in improving L2 written accuracy: a meta-analysis. Mod. Lang. J. 99(1), 1–18 (2015)

    Article  Google Scholar 

  24. Koh, W.: Effective applications of automated writing feedback in process-based writing instruction. Engl. Teach. 72(3), 91–118 (2017)

    Google Scholar 

  25. Biber, D., Nekrasova, T., Horn, B.: The effectiveness of feedback for L1-English and L2-Writing Development: A meta-analysis (2011)

    Article  Google Scholar 

  26. Ferris, D.R.: Does error feedback help student writers? New evidence on the short-and long-term effects of written error correction. In: Hyland, K., Hyland, F. (eds.) Feedback in Second Language Writing: Contexts and Issues, pp 81–104. Cambridge University Press, Cambridge (2006)

    Google Scholar 

  27. Zamel, V.: Responding to student writing. TESOL Q. 19(1), 79–101 (1985)

    Article  Google Scholar 

  28. Li, J., Link, S., Hegelheimer, V.: Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. J. Second Lang. Writ. 27, 1–18 (2015)

    Article  Google Scholar 

  29. Tang, J., Rich, C.S.: Automated writing evaluation in an EFL setting: lessons from China. JALT CALL J. 13(2), 117–146 (2017)

    Google Scholar 

  30. Wang, M., Goodman, D.: Automated writing evaluation: students’ perceptions and emotional involvement. Engl. Teach. Learn. 36(3), 1–37 (2012)

    Google Scholar 

  31. Cotos, E., Huffman, S.: Learner fit in scaling up automated writing evaluation. Int. J. Comput. Assist. Lang. Learn. Teach. 3(3), 77–98 (2013)

    Article  Google Scholar 

  32. Wang, Y.-J., Shang, H.-F., Briody, P.: Exploring the impact of using automated writing evaluation in English as a foreign language university students’ writing. Comput. Assist. Lang. Learn. 26(3), 234–257 (2013)

    Article  Google Scholar 

  33. Liao, H.C.: Enhancing the grammatical accuracy of EFL writing by using an AWE-assisted process approach. System 62, 77–92 (2016)

    Article  Google Scholar 

  34. Hyland, K.: Second Language Writing. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  35. Van Beuningen, C.: Corrective feedback in L2 writing: theoretical perspectives, empirical insights and future directions. Int. J. Engl. Stud. 10(2), 1–27 (2010)

    Article  Google Scholar 

  36. Bartolome, A., Steffens, K.: Technologies for self-regulated learning. In: Carneiro, R. (ed.) Self-Regulated Learning in Technology Enhanced Learning Environments, pp. 21–31. Sense, Rotterdam (2011)

    Chapter  Google Scholar 

  37. Winne, P.H.: How software technologies can improve research on learning and Bolster School Reform. Educ. Psychol. 41(1), 5–17 (2006)

    Article  MathSciNet  Google Scholar 

  38. Saadi, Z.K., Saadat, M.: EFL learners’ writing accuracy: effects of direct and metalinguistic electronic feedback. Theory Pract. Lang. Stud. 5(10), 2053–2063 (2015)

    Article  Google Scholar 

  39. Polio, C.: The relevance of second language acquisition theory to the written error correction debate. J. Second Lang. Writ. 21, 375–389 (2012)

    Article  Google Scholar 

  40. Li, Z., Feng, H., Saricaoglu, A.: The short-term and long-term effects of AWE feedback on ESL students’ development of grammatical accuracy. CALICO J. 34(3), 355–375 (2017)

    Google Scholar 

  41. Hartshorn, K.J., Evans, N.W., Merril, P.F., et al.: Effects of dynamic corrective feedback on ESL writing accuracy. TESOL Q. 44(1), 84–109 (2010)

    Article  Google Scholar 

  42. Zaini, A., Mazdayasna, G.: The impact of computer-based instruction on the development of EFL learners’ writing skills. J. Comput. Assist. Learn. 31, 516–528 (2015)

    Article  Google Scholar 

  43. Hattie, J.: Influences on student learning. In: Inaugural Lecture: University of Auckland, pp 1–25 (1999)

    Google Scholar 

  44. Gašević, D., Mirriahi, N., Dawson, S., Joksimović, S.: Effects of instructional conditions and experience on the adoption of a learning tool. Comput. Hum. Behav. 67, 207–220 (2017)

    Article  Google Scholar 

  45. Kitsantas, A.: Fostering college students’ self-regulated learning with learning technologies. Hellenic J. Psychol. 10, 235–252 (2013)

    Google Scholar 

  46. Lust, G., Juarez Collazo, N.A., Elen, J., Clarebout, G.: Content management systems: enriched learning opportunities for all? Comput. Hum. Behav. 28(3), 795–808 (2012)

    Article  Google Scholar 

  47. Hattie, J., Timperley, H.: The power of feedback. Rev. Educ. Res. 77(1), 81–112 (2007)

    Article  Google Scholar 

  48. Liu, Q., Brown, D.: Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. J. Second Lang. Writ. 30, 66–81 (2015)

    Article  Google Scholar 

  49. Grami, G.M.A., Alkazemi, B.Y.: Improving ESL writing using an online formulaic sequence word-combination checker. J. Comput. Assist. Learn. 32, 95–104 (2016)

    Article  Google Scholar 

  50. Guénette, D.: Is feedback pedagogically correct? Research design issues in studies of feedback on writing. J. Second Lang. Writ. 16, 40–53 (2007)

    Article  Google Scholar 

  51. Dugard, P., Todman, J.: Analysis of pre-test-post-test control group designs in educational research. Educ. Psychol. 15(2), 181–198 (1995)

    Article  Google Scholar 

  52. Venkatesh, V., Davis, F.D.: A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag. Sci. 46(2), 186–204 (2000)

    Article  Google Scholar 

  53. Ortega, L.: Syntactic complexity measures and their relationship to L2 proficiency: a research synthesis of college-level L2 writing. Appl. Linguist. 24(4), 492–518 (2003)

    Article  Google Scholar 

  54. Graesser, A.C., McNamara, D., Louwerse, M.M., Cai, Z.: Coh-metrix: analysis of text on cohesion and language. Behav. Res. Methods Instrum. Comput. 36(2), 193–202 (2004)

    Article  Google Scholar 

  55. Lu, X.: Automatic analysis of syntactic complexity in second language writing. Int. J. Corpus Linguist. 15(4), 474–496 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Edinburgh, Edinburgh, EH8 8AJ, UK

    Ana Isabel Hibert

Authors
  1. Ana Isabel Hibert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Isabel Hibert .

Editor information

Editors and Affiliations

  1. Open University Netherlands, Heerlen, The Netherlands

    Maren Scheffel

  2. Paul Sabatier University, Toulouse, France

    Julien Broisin

  3. Know-Center GmbH, Graz, Austria

    Viktoria Pammer-Schindler

  4. Cyprus University of Technology, Limassol, Cyprus

    Andri Ioannou

  5. DIPF, Frankfurt/Main, Germany

    Jan Schneider

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hibert, A.I. (2019). Systematic Literature Review of Automated Writing Evaluation as a Formative Learning Tool. In: Scheffel, M., Broisin, J., Pammer-Schindler, V., Ioannou, A., Schneider, J. (eds) Transforming Learning with Meaningful Technologies. EC-TEL 2019. Lecture Notes in Computer Science(), vol 11722. Springer, Cham. https://doi.org/10.1007/978-3-030-29736-7_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29736-7_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29735-0

  • Online ISBN: 978-3-030-29736-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation