skip to main content
10.1145/3498851.3498983acmconferencesArticle/Chapter ViewAbstractPublication PageswiConference Proceedingsconference-collections
research-article

Research on A Machine Scoring Method of Role-Play Section in English Oral Test

Published: 11 April 2022 Publication History

Abstract

Computer-assisted instruction has been widely implemented in language learning since the remarkable development of speech technology and natural language processing. In English teaching, the traditional manual evaluation mode has been replaced by the computer. In this paper, we introduce a machine scoring method of Role-Play Section in the English oral test. Specifically, we design a scoring method of Role-Play section in Computer-based English Listening and Speaking Test (CELST) of the Guangdong college entrance examination (GDCEE), which can simulate human scoring. The role-play section is also called Three Questions (TQ) and Five Answers (FA), so two scoring modules are established, respectively. According to the features of the role-play section and the given manual scoring scale, the scoring task can be regarded as a short text similarity evaluation task, and we propose a corresponding multi-index text similarity evaluation method. Statistic-based word matching and keyword-focused semantic similarity evaluations are adopted in the TQ and FA scoring modules. Additionally, we consider the grammar factor and utilize semantic similarity-combined dependency parsing evaluation for TQ scoring. Based on the above five evaluative indicators, we linearly integrate them and corresponding expertise-based weights with linear regression optimization, thus constructing a comprehensive scoring model of the role-play Section in CELST. Experiments were conducted using real oral data of examinees in GDCEE. Results have indicated that the machine scoring method achieves impressive data consistency with human scoring.

References

[1]
Hua Chen and Jingna Li. 2017. Reflections on the current situation of phonological assessment: a survey among raters of standardized oral tests in China. Foreign Languages and Their Teaching, (05), 81-87+131+149
[2]
Hua Chen, Kui Wu and Jingna Li. 2019. The automatic scoring system of Chinese Learners’ spoken English—a new system to assess oral English proficiency. Technology Enhanced Foreign Language Education, (01):72-77.
[3]
Xia Gao. 2007. Commentary and design of the scoring scheme for reading aloud. Foreign Language Learning Theory and Practice, (4), 15-20.
[4]
Yulin Wang, Fan Guo and Min Yu. 2013. Research on pronunciation errors in automatic oral English scoring system. Computer Applications and Software, 30(06):214-217.
[5]
Ke Yan. 2009. Research on automatic evaluation of English recitation and retelling test [D]. University of Science and Technology of China.
[6]
Ke Yan, Guoping Hu, Si Wei, Lirong Dai, Mengtao Li, Xiaoguo Yang and Guodong Feng. 2009. Automatic evaluation of English retelling proficiency in large machine based oral English tests. Journal of Tsinghua University (Science and Technology), (49):1356-1362.
[7]
Haiyang Sun. 2021. Automated scoring of speech: an overview of research at home and abroad. Foreign Language Education in China, 24(02):28-36.
[8]
Yandong Zhai, Kangping Wang, Dongna Zhang, Lan Huang and Chunguang Zhou. 2012. An algorithm for semantic similarity of short text based on WordNet. Acta Electronica Sinica, 40(03): 617-620.
[9]
Roy Rada, Hafedh Mili, Ellen Bicknell and Maria Blettner. 1989. Development and application of a metric on semantic nets. IEEE Transactions on Systems, Man, and Cybernetics, 1989,19(1):17 -30.
[10]
Zhibiao Wu and Martha Palmer. 1994. Verb Semantics and Lexical Selection[C]. In: Proceedings ofthe32nd Annual Meeting of the Associations for Computational Linguistics. 133 -138.
[11]
Robert Stevens, Andy Brass, Carole Goble and Phillip W. Lord. 2003. Investigating Semantic Similarity Measures Across the Gene Ontology: The Relationship Between Sequence and Annotation. Bioinformatics,19(10):1275 -1283.
[12]
Dekang Lin. 1993. Principle-based Parsing Without Overgeneration[C]. In: Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics. 112 -120.
[13]
Jay J. Jiang and David W. Conrath. 2008. Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy [ EB/OL]. http://arxiv.org/PS_cache/cmp -lg/pdf/9709 /9709008v1.pdf.
[14]
Amos Tversky. 1997. Features of Similarity. Psycological Review, 84(4):327 -352.
[15]
Zhe Hu and Cheng Zheng. 2010. Improved concept similarity computation. Computer Engineering and Design, 31(05):1121-1124.
[16]
Qiwen Cui and Fu Xie. 2012. An improved computational method for conceptual semantic similarity in domain ontology. Computer Applications and Software, 29(02):173-174.
[17]
R. Richardson, Alan F. Smeaton and J. Murphy J. 2008. Using WordNet as a Knowledge Base for Measuring Semantic Similarity Between Words [EB/OL]. In Proceedings of AICS Conference. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.124.4773&rep=rep1&type=pdf.
[18]
Claudia Leacock and Martin Chodorow. 2009. Filling in a Sparse Training Space for Word Sense Identification [EB/OL]. http://arxiv.org/PS_cache/cmp%20-lg/pdf/9511%20/9511007v1.pdf.
[19]
Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel and Steven Bethard. 2014. The Stanford CoreNLP Natural Language Processing Toolkit In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55-60.

Cited By

View all
  • (2022)Cultural Capital and Its Impact on Academic Achievement: Sustainable Development of Chinese High School StudentsSustainability10.3390/su14221497614:22(14976)Online publication date: 12-Nov-2022
  1. Research on A Machine Scoring Method of Role-Play Section in English Oral Test

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      WI-IAT '21: IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology
      December 2021
      541 pages
      ISBN:9781450391870
      DOI:10.1145/3498851
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 11 April 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      WI-IAT '21
      Sponsor:
      WI-IAT '21: IEEE/WIC/ACM International Conference on Web Intelligence
      December 14 - 17, 2021
      VIC, Melbourne, Australia

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)7
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 17 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2022)Cultural Capital and Its Impact on Academic Achievement: Sustainable Development of Chinese High School StudentsSustainability10.3390/su14221497614:22(14976)Online publication date: 12-Nov-2022

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media