Skip to main content
Springer Nature Link
Log in

Enhanced Hierarchical Structure Features for Automated Essay Scoring

  • Conference paper
  • First Online:
Information Retrieval (CCIR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 13026))

Included in the following conference series:

Abstract

Automated Essay Scoring (AES) aims to evaluate the quality of an essay automatically. In practice, an essay is usually organized in a hierarchical structure, which means that the writer needs to organize the main ideas into different paragraphs, and organize coherent sentences and appropriate words for the essay. Therefore, it is crucial to model the hierarchical structure of essays for AES. For addressing this issue, most of the existing works used neural network-based architectures (e.g., CNNs and LSTMs) to model the hierarchical structure of essays. Different from previous studies, we propose a novel hierarchical graph structure based on graph convolutional networks (GCN) to encode the hierarchical structure of essays and hope to obtain those structured coherence and discourse information from the graph aggregation. We conduct several experiments on ASAP dataset and the experimental results demonstrate the effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.kaggle.com/c/asap-aes/data.

References

  1. Alikaniotis, D., Yannakoudakis, H., Rei, M.: Automatic text scoring using neural networks. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (2016)

    Google Scholar 

  2. Ba, L.J., Kiros, J.R., Hinton, G.E.: Layer normalization. CoRR abs/1607.06450 (2016). arXiv:1607.06450

  3. Chen, H., He, B.: Automated essay scoring by maximizing human-machine agreement. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1741–1752 (2013)

    Google Scholar 

  4. Dauphin, Y.N., de Vries, H., Bengio, Y.: Equilibrated adaptive learning rates for non-convex optimization. In: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1. NIPS 2015, pp. 1504–1512. MIT Press (2015)

    Google Scholar 

  5. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT 2019, pp. 4171–4186. Association for Computational Linguistics (2019)

    Google Scholar 

  6. Dong, F., Zhang, Y.: Automatic features for essay scoring - an empirical study. In: Proceedings of EMNLP 2016, pp. 1072–1077 (2016)

    Google Scholar 

  7. Dong, F., Zhang, Y., Yang, J.: Attention-based recurrent convolutional neural network for automatic essay scoring. In: Proceedings of CoNLL 2017, pp. 153–162 (2017)

    Google Scholar 

  8. Farag, Y., Yannakoudakis, H., Briscoe, T.: Neural automated essay scoring and coherence modeling for adversarially crafted input. In: Proceedings of NAACL-HLT 2018, pp. 263–271 (2018)

    Google Scholar 

  9. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

    Google Scholar 

  10. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. CoRR abs/1609.02907 (2016). arXiv:1609.02907

  11. Larkey, L.S.: Automatic essay grading using text categorization techniques. In: Proceedings of the 21st Annual International Conference on Research and Development in Information Retrieval, Melbourne, Australia, 24–28 August 1998, pp. 90–95 (1998)

    Google Scholar 

  12. Li, X., Chen, M., Nie, J., Liu, Z., Feng, Z., Cai, Y.: Coherence-based automated essay scoring using self-attention. In: Sun, M., Liu, T., Wang, X., Liu, Z., Liu, Y. (eds.) CCL/NLP-NABD -2018. LNCS (LNAI), vol. 11221, pp. 386–397. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01716-3_32

    Chapter  Google Scholar 

  13. Liao, D., Xu, J., Li, G., Wang, Y.: Hierarchical coherence modeling for document quality assessment. In: Proceedings of the AAAI 2021, vol. 35, no. 15, pp. 13353–13361, May 2021

    Google Scholar 

  14. van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(86), 2579–2605 (2008)

    MATH  Google Scholar 

  15. Mayfield, E., Black, A.W.: Should you fine-tune BERT for automated essay scoring? In: Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, BEA@ACL 2020, pp. 151–162. Association for Computational Linguistics (2020)

    Google Scholar 

  16. McNamara, D., Crossley, S., Roscoe, R., Allen, L.: A hierarchical classification approach to automated essay scoring. Assess. Writ. 23, 35–59 (2015)

    Article  Google Scholar 

  17. Ormerod, C.M., Malhotra, A., Jafari, A.: Automated essay scoring using efficient transformer-based language models. CoRR abs/2102.13136 (2021). https://arxiv.org/abs/2102.13136

  18. Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of EMNLP 2014, pp. 1532–1543. Association for Computational Linguistics, October 2014

    Google Scholar 

  19. Rudner, L.M., Liang, T.: Automated essay scoring using Bayes’ theorem. J. Technol. Learn. Assess. 1(2), 1–22 (2002)

    Google Scholar 

  20. Seo, M.J., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional attention flow for machine comprehension. In: Proceedings of ICLR 2017. OpenReview.net (2017)

    Google Scholar 

  21. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, pp. 3104–3112 (2014)

    Google Scholar 

  22. Taghipour, K., Ng, H.T.: A neural approach to automated essay scoring. In: Proceedings of EMNLP 2016, pp. 1882–1891 (2016)

    Google Scholar 

  23. Tay, Y., Phan, M.C., Tuan, L.A., Hui, S.C.: SkipFlow: incorporating neural coherence features for end-to-end automatic text scoring. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pp. 5948–5955 (2018)

    Google Scholar 

  24. Uto, M., Xie, Y., Ueno, M.: Neural automated essay scoring incorporating handcrafted features. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 6077–6088. International Committee on Computational Linguistics, December 2020

    Google Scholar 

  25. Vaswani, A., et al.: Attention is all you need. In: Proceedings of NIPS 2017. NIPS 2017, pp. 6000–6010. Curran Associates Inc. (2017)

    Google Scholar 

  26. Wang, Y., Wei, Z., Zhou, Y., Huang, X.: Automatic essay scoring incorporating rating schema via reinforcement learning. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 791–797. Association for Computational Linguistics, October-November 2018

    Google Scholar 

  27. Wang, Y., Wei, Z., Zhou, Y., Huang, X.: Automatic essay scoring incorporating rating schema via reinforcement learning. In: Proceedings of EMNLP 2018, pp. 791–797 (2018)

    Google Scholar 

  28. Yang, R., Cao, J., Wen, Z., Wu, Y., He, X.: Enhancing automated essay scoring performance via fine-tuning pre-trained language models with combination of regression and ranking. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1560–1569. Association for Computational Linguistics, November 2020

    Google Scholar 

  29. Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: The Thirty-Third AAAI Conference on Artificial Intelligence. AAAI 2019, pp. 7370–7377. AAAI Press (2019)

    Google Scholar 

  30. Zhang, C., Li, Q., Song, D.: Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of EMNLP-IJCNLP 2019, pp. 4567–4577. Association for Computational Linguistics (2019)

    Google Scholar 

  31. Zhang, H., Litman, D.J.: Co-attention based neural network for source-dependent essay scoring. In: Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications@NAACL-HLT 2018, pp. 399–409. Association for Computational Linguistics (2018)

    Google Scholar 

  32. Zhao, S., Zhang, Y., Xiong, X., Botelho, A., Heffernan, N.T.: A memory-augmented neural model for automated grading. In: Proceedings of the Fourth ACM Conference on Learning @ Scale, L@S 2017, pp. 189–192 (2017)

    Google Scholar 

Download references

Acknowledgements

This work is supported by National Nature Science Foundation of China (61976062) and Special Funds for the Cultivation of Guangdong College Students’ Scientific and Technological Innovation (pdjh2021b0177).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Guangdong University of Foreign Studies, Guangzhou, China

    Junteng Ma, Xia Li, Minping Chen & Weigeng Yang

  2. Guangzhou Key Laboratory of Multilingual Intelligent Processing, Guangdong University of Foreign Studies, Guangzhou, China

    Xia Li

Authors
  1. Junteng Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xia Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minping Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weigeng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xia Li .

Editor information

Editors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Hongfei Lin

  2. Department of Computer Science, Tsinghua University, Beijing, China

    Min Zhang

  3. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Liang Pang

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ma, J., Li, X., Chen, M., Yang, W. (2021). Enhanced Hierarchical Structure Features for Automated Essay Scoring. In: Lin, H., Zhang, M., Pang, L. (eds) Information Retrieval. CCIR 2021. Lecture Notes in Computer Science(), vol 13026. Springer, Cham. https://doi.org/10.1007/978-3-030-88189-4_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88189-4_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88188-7

  • Online ISBN: 978-3-030-88189-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics