Skip to main content
SpringerLink
Log in

Feature Group Importance for Automated Essay Scoring

  • Conference paper
  • First Online:
Multi-disciplinary Trends in Artificial Intelligence (MIWAI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12832))

Abstract

One of the challenges in essay scoring is that it is highly subjective to the human graders. There have been numerous research projects conducted on improving computerised Automated Essay Scoring (AES). AES systems generally rely on hand-crafted linguistic features to construct a classification model for essay scoring. The majority of the AES systems’ classification algorithm inputs are based on three main feature groups; lexical, grammatical, and semantic feature groups. This paper presents an empirical study to explore the influence of each feature group on the performance of AES classification models based on a general approach of the AES system. The results uncovered that the grammatical and semantic feature groups are lacking due to their poor performance and typical over-fitting of the classification models when using the features in the feature group.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Attali, Y., Burstein, J.: Automated essay scoring with E-rater® v. 2. J. Technol. Learn. Assess. 4(3) (2006)

    Google Scholar 

  2. Bisong, E.: Building Machine Learning and Deep Learning Models on Google Cloud Platform. Apress, Berkeley, CA (2019). https://doi.org/10.1007/978-1-4842-4470-8

    Book  Google Scholar 

  3. Briscoe, T., Carroll, J.A., Watson, R.: The second release of the rasp system. In: Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions, pp. 77–80 (2006)

    Google Scholar 

  4. Chen, H., He, B.: Automated essay scoring by maximizing human-machine agreement. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1741–1752 (2013)

    Google Scholar 

  5. Christie, J.R.: Automated essay marking-for both style and content. In: Proceedings of the Third Annual Computer Assisted Assessment Conference, Loughborough University, Loughborough, UK. Citeseer (1999)

    Google Scholar 

  6. Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20(1), 37–46 (1960)

    Article  Google Scholar 

  7. Cozma, M., Butnaru, A.M., Ionescu, R.T.: Automated essay scoring with string kernels and word embeddings. arXiv preprint arXiv:1804.07954 (2018)

  8. Eid, S.M., Wanas, N.M.: Automated essay scoring linguistic feature: comparative study. In: 2017 International Conference on Advanced Control Circuits Systems (ACCS) Systems & 2017 International Conference on New Paradigms in Electronics & Information Technology (PEIT), pp. 212–217. IEEE (2017)

    Google Scholar 

  9. Foltz, P.W., Laham, D., Landauer, T.K.: The intelligent essay assessor: applications to educational technology. Interact. Multimedia Electron. J. Comput. Enhanced Learn. 1(2), 939–944 (1999)

    Google Scholar 

  10. Graesser, A.C., McNamara, D.S., Louwerse, M.M., Cai, Z.: Coh-Metrix: analysis of text on cohesion and language. Behav. Res. Methods Instrum. Comput. 36(2), 193–202 (2004)

    Article  Google Scholar 

  11. Janda, H.K., Pawar, A., Du, S., Mago, V.: Syntactic, semantic and sentiment analysis: the joint effect on automated essay evaluation. IEEE Access 7, 108486–108503 (2019)

    Article  Google Scholar 

  12. Latifi, S., Gierl, M.: Automated scoring of junior and senior high essays using coh-metrix features: implications for large-scale language testing. Lang. Test. (2020). https://doi.org/10.1177/0265532220929918

  13. Liu, H., Ye, Y., Wu, M.: Ensemble learning on scoring student essay. In: 2018 International Conference on Management and Education, Humanities and Social Sciences (MEHSS 2018), pp. 250–255. Atlantis Press (2018)

    Google Scholar 

  14. Liu, J., Xu, Y., Zhu, Y.: Automated essay scoring based on two-stage learning. arXiv preprint arXiv:1901.07744 (2019)

  15. McNamara, D.S., Louwerse, M.M., McCarthy, P.M., Graesser, A.C.: Coh-Metrix: capturing linguistic features of cohesion. Discourse Process. 47(4), 292–330 (2010)

    Article  Google Scholar 

  16. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. arXiv preprint arXiv:1310.4546 (2013)

  17. Miltsakaki, E., Kukich, K.: Automated evaluation of coherence in student essays. In: Proceedings of LREC, pp. 1–8 (2000)

    Google Scholar 

  18. Nguyen, H., Litman, D.: Argument mining for improving the automated scoring of persuasive essays. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  19. Norton, L.S.: Essay-writing: what really counts? High. Educ. 20(4), 411–442 (1990)

    Article  Google Scholar 

  20. Ormerod, C.M., Malhotra, A., Jafari, A.: Automated essay scoring using efficient transformer-based language models. arXiv preprint arXiv:2102.13136 (2021)

  21. Page, E.B.: The imminence of... grading essays by computer. Phi Delta Kappan 47(5), 238–243 (1966)

    Google Scholar 

  22. Phandi, P., Chai, K.M.A., Ng, H.T.: Flexible domain adaptation for automated essay scoring using correlated linear regression. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 431–439 (2015)

    Google Scholar 

  23. Sharaff, A., Gupta, H.: Extra-tree classifier with metaheuristics approach for email classification. In: Bhatia, S.K., Tiwari, S., Mishra, K.K., Trivedi, M.C. (eds.) Advances in Computer Communication and Computational Sciences. AISC, vol. 924, pp. 189–197. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-6861-5_17

    Chapter  Google Scholar 

  24. Shawe-Taylor, J., Cristianini, N., et al.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  25. Shermis, M.D., Burstein, J.C.: Automated Essay Scoring: A Cross-disciplinary Perspective. Routledge (2003)

    Google Scholar 

  26. Vanbelle, S., Albert, A.: A note on the linearly weighted kappa coefficient for ordinal scales. Stat. Methodol. 6(2), 157–163 (2009)

    Article  MathSciNet  Google Scholar 

  27. Yang, R., Cao, J., Wen, Z., Wu, Y., He, X.: Enhancing automated essay scoring performance via cohesion measurement and combination of regression and ranking. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp. 1560–1569 (2020)

    Google Scholar 

  28. Yannakoudakis, H., Briscoe, T., Medlock, B.: A new dataset and method for automatically grading ESOL texts. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 180–189 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Priority Dynamics Sdn Bhd, E-05-01 & 02, Garden Shoppe, Jalan USJ 25/1C, One City, 47650, Subang Jaya, Selangor, Malaysia

    Jih Soong Tan

  2. Monash University Malaysia, Bandar Sunway, 47500, Subang Jaya, Malaysia

    Ian K. T. Tan

Authors
  1. Jih Soong Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ian K. T. Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jih Soong Tan .

Editor information

Editors and Affiliations

  1. Mahasarakham University, Maha Sarakham, Thailand

    Phatthanaphong Chomphuwiset

  2. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Junmo Kim

  3. Mahasarakham University, Maha Sarakham, Thailand

    Pornntiwa Pawara

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tan, J.S., Tan, I.K.T. (2021). Feature Group Importance for Automated Essay Scoring. In: Chomphuwiset, P., Kim, J., Pawara, P. (eds) Multi-disciplinary Trends in Artificial Intelligence. MIWAI 2021. Lecture Notes in Computer Science(), vol 12832. Springer, Cham. https://doi.org/10.1007/978-3-030-80253-0_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-80253-0_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-80252-3

  • Online ISBN: 978-3-030-80253-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation