Skip to main content
SpringerLink
Log in

Automated Scoring of Logical Consistency of Japanese Essays

  • Conference paper
  • First Online:
Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky (AIED 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1831))

Included in the following conference series:

  • 2654 Accesses

Abstract

Automated essay scoring (AES) is to estimate the scores of essays automatically. Two types of AES models are commonly used: handcrafted feature-based and neural-based models. In this paper, we introduce AES systems based on the two types for evaluating the logical consistency of Japanese essays. In addition, to enhance the performance of models, we integrate the neural-based model with the handcrafted features: a hybrid AES system. In the experiment, we show the effectiveness of our hybrid AES system. Besides, most of our AES models obtained higher QWK scores than human evaluators.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://gsk.or.jp/catalog/gsk2021-b (in Japanese).

  2. 2.

    Although we evaluated other regression models such as SVR, Random Forest Regressor tended to be better than them.

  3. 3.

    https://github.com/cl-tohoku/bert-japanese.

References

  1. Attali, Y., Burstein, J.: Automated essay scoring with e-rater vol 2. J. Technol. Learn. Assess. 4(3) (2006)

    Google Scholar 

  2. Burstein, J., et al.: Automated scoring using a hybrid feature identification technique. In: 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, vol, 1, pp. 206–210 (1998)

    Google Scholar 

  3. Cohen, J.: Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol. Bull. 70(4), 213–220 (1968)

    Article  Google Scholar 

  4. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 4171–4186 (2019). https://doi.org/10.18653/v1/N19-1423

  5. Ishioka, T., Kameda, M.: Automated Japanese essay scoring system based on articles written by experts. In: Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pp. 233–240 (2006)

    Google Scholar 

  6. Kiyono, M., Koichi, T.: Automatic scoring of Japanese essay using neural networks. Forum Inf. Technol. 18(2), 239–240 (2019). (In Japanese)

    Google Scholar 

  7. Ohno, M., et al.: Construction of open essay writing data and automatic essay scoring system for Japanese. In: Proceedings of The 15th International Conference of the Pacific Association for Computational Linguistics, vol. 17, pp. 215–220 (2017)

    Google Scholar 

  8. Uto, M., Xie, Y., Ueno, M.: Neural automated essay scoring incorporating handcrafted features. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 6077–6088 (2020)

    Google Scholar 

  9. Yule, C.U.: The Statistical Study of Literary Vocabulary. Cambridge University Press, Cambridge (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kyushu Institute of Technology, Iizuka, Japan

    Sayaka Nakamoto & Kazutaka Shimada

Authors
  1. Sayaka Nakamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kazutaka Shimada
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sayaka Nakamoto .

Editor information

Editors and Affiliations

  1. University of Southern California, Los Angeles, CA, USA

    Ning Wang

  2. University of British Columbia, Vancouver, BC, Canada

    Genaro Rebolledo-Mendez

  3. University of Leeds, Leeds, UK

    Vania Dimitrova

  4. North Carolina State University, Raleigh, NC, USA

    Noboru Matsuda

  5. UNED, Madrid, Spain

    Olga C. Santos

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nakamoto, S., Shimada, K. (2023). Automated Scoring of Logical Consistency of Japanese Essays. In: Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., Santos, O.C. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2023. Communications in Computer and Information Science, vol 1831. Springer, Cham. https://doi.org/10.1007/978-3-031-36336-8_101

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-36336-8_101

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-36335-1

  • Online ISBN: 978-3-031-36336-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation