Abstract
Second language (L2) writing plays an important role in improving the learners’ language skills of English as a Foreign Language (EFL) in terms of language expression and linguistic thinking. Therefore, improving writing skills is still a focus area for EFL learners. To enhance EFL learners’ writing ability and optimize their writing quality, an intelligent-based cognitive diagnostic feedback (I-CDF) strategy is proposed based on the Intelligent Writing Critique System (IWCS). IWCS can provide feedback on students’ English writing learning components, including lexical, syntactic, rhetorical expression, chapter structure, and discourse intention. Hence, the study intends to assess the effects of feedback strategies based on students’ writing scores, self-efficacy, epistemic network structure, and transferability. A quasi-experiment was conducted in two classes at a university in southeastern China, where students were randomly divided into two classes. One (N = 32) was considered an experimental group conducted using the proposed I-CDF strategy, while the other (N = 30) was the control group using the score-based teacher corrective feedback (S-TCF) strategy. The writing experiment lasted for seven weeks. The students would be interviewed at the end of the writing learning activities. The results indicated that the I-CDF strategy improved students’ writing scores and self-efficacy. Furthermore, the epistemic network analysis showed that, compared to the control group, the I-CDF strategy encouraged the students to devote more energy to focus on high-level applications of writing skills such as rhetorical expression and sentence structure collation, optimizing the students’ writing epistemic network of the writing of the students. The interview revealed that the I-CDF strategy supported the experimental group students’ accurate understanding of writing, strengthening the logical structure of the writing. At the same time, the students in the experimental group were satisfied with the I-CDF strategy.
Similar content being viewed by others
Data availability
The data and materials are available upon request to the corresponding author.
Code availability
Not applicable.
References
Bai, L., & Hu, G. (2017). In the face of fallible AWE feedback: How do students respond? Educational Psychology, 37(1), 67–81. https://doi.org/10.1080/01443410.2016.1223275
Bandura, A. (1986). The explanatory and predictive scope of self-efficacy theory. Journal of Social and Clinical Psychology, 4(3), 359. https://doi.org/10.1521/jscp.1986.4.3.359
Bandura, A. (1995). Comments on the crusade against the causal efficacy of human thought. Journal of Behavior Therapy and Experimental Psychiatry, 26(3), 179–190. https://doi.org/10.1016/0005-7916(95)00034-W
Bandura, A. (1997). Self-efficacy: The exercise of control. Macmillan.
Binder, K. S., Cote, N. G., Lee, C., Bessette, E., & Vu, H. (2017). Beyond breadth: The contributions of vocabulary depth to reading comprehension among skilled readers. Journal of Research in Reading, 40(3), 333–343. https://doi.org/10.1111/1467-9817.12069
Birenbaum, M., Kelly, A. E., & Tatsuoka, K. K. (1993). Diagnosing knowledge states in algebra using the rule-space model. Journal for Research in Mathematics Education, 24(5), 442–459. https://doi.org/10.5951/jresematheduc.24.5.0442
Brüggemann, T., Ludewig, U., Lorenz, R., & McElvany, N. (2023). Effects of mode and medium in reading comprehension tests on cognitive load. Computers & Education, 192, 104649. https://doi.org/10.1016/j.compedu.2022.104649
Bruning, R., Dempsey, M., Kauffman, D. F., McKim, C., & Zumbrunn, S. (2013). Examining dimensions of self-efficacy for writing. Journal of Educational Psychology, 105(1), 25. https://doi.org/10.1037/a0029692
Carless, D. (2015). Exploring learning-oriented assessment processes. Higher Education, 69(6), 963–976. https://doi.org/10.1007/s10734-014-9816-z
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354
Chang, C. Y. H. (2015). Teacher modeling on EFL reviewers’ audience-aware feedback and affectivity in L2 peer review. Assessing Writing, 25, 2–21. https://doi.org/10.1016/j.asw.2015.04.001
Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Sage.
Cheville, J. (2004). Automated scoring technologies and the rising influence of error. The English Journal, 93(4), 47–52. https://doi.org/10.2307/4128980
Creswell, J. W. (2013). Steps in conducting a scholarly mixed methods study. DBER Speaker Series.
Csanadi, A., Eagan, B., Kollar, I., Shaffer, D. W., & Fischer, F. (2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13, 419–438. https://doi.org/10.1007/s11412-018-9292-z
DePalma, M. J., & Ringer, J. M. (2011). Toward a theory of adaptive transfer: Expanding disciplinary discussions of “transfer” in second-language writing and composition studies. Journal of Second Language Writing, 20(2), 134–147. https://doi.org/10.1016/j.jslw.2011.02.003
Dikli, S., & Bleyle, S. (2014). Automated essay scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17. https://doi.org/10.1016/j.asw.2014.03.006
Elmoazen, R., Saqr, M., Tedre, M., & Hirsto, L. (2022). A systematic literature review of empirical research on epistemic network analysis in education. IEEE Access. https://doi.org/10.1109/ACCESS.2022.3149812
Engber, C. A. (1995). The relationship of lexical proficiency to the quality of ESL compositions. Journal of Second Language Writing, 4(2), 139–155. https://doi.org/10.1016/1060-3743(95)90004-7
Evans, C. (2013). Making sense of assessment feedback in higher education. Review of Educational Research, 83(1), 70–120. https://doi.org/10.3102/0034654312474350
Fougt, S. S., Siebert-Evenstone, A., Eagan, B., Tabatabai, S., & Misfeldt, M. (2018). Epistemic network analysis of students’ longer written assignments as formative/summative evaluation. In Proceedings of the 8th international conference on learning analytics and knowledge. https://doi.org/10.1145/3170358.3170414.
Gao, J. (2021). Exploring the feedback quality of an automated writing evaluation system pigai. International Journal of Emerging Technologies in Learning (iJET), 16(11), 322–330. https://doi.org/10.3991/ijet.v16i11.19657
Gao, Y., Zhai, X., Andersson, B., Zeng, P., & Xin, T. (2020). Developing a learning progression of buoyancy to model conceptual change: A latent class and rule space model analysis. Research in Science Education, 50(4), 1369–1388. https://doi.org/10.1007/s11165-018-9736-5
Gardner, D., & Davies, M. (2014). A new academic vocabulary list. Applied Linguistics, 35(3), 305–327. https://doi.org/10.1093/applin/amt015
Gibbs, G., & Simpson, C. (2004). Does your assessment support your students’ learning. Journal of Teaching and learning in Higher Education, 1(1), 1–30.
Glaser, B. G., & Strauss, A. L. (2017). The discovery of grounded theory: Strategies for qualitative research. Routledge. https://doi.org/10.4324/9780203793206
Goldstone, R. L., & Day, S. B. (2012). Introduction to “new conceptualizations of transfer of learning”. Educational Psychologist, 47(3), 149–152. https://doi.org/10.1080/00461520.2012.695710
Graham, S., & Hebert, M. (2011). Writing to read: A meta-analysis of the impact of writing and writing instruction on reading. Harvard Educational Review, 81(4), 710–744. https://doi.org/10.17763/haer.81.4.t2k0m13756113566
Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. The Journal of Technology, Learning and Assessment, 8(6).
Hansen, J. G. (2000). Interactional conflicts among audience, purpose, and content knowledge in the acquisition of academic literacy in an EAP course. Written Communication, 17(1), 27–52. https://doi.org/10.1177/0741088300017001002
Hansen, E. G., Shute, V. J., & Landau, S. (2010). An assessment-for-learning system in mathematics for individuals with visual impairments. Journal of Visual Impairment & Blindness, 104(5), 275–286. https://doi.org/10.1177/0145482X1010400503
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
Hayes, I. R. (2000). A new framework for understanding cognition and affect in writing. In R. Indrisano & J. R. Squire (Eds.), Perspectives on writing: Research, theory, and practice. International Reading Association.
James, M. A. (2012). An investigation of motivation to transfer second language learning. The Modern Language Journal, 96(1), 51–69. https://doi.org/10.1111/j.1540-4781.2012.01281.x
Jang, E. E. (2009). Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for fusion model application to LanguEdge assessment. Language Testing, 26(1), 031–073. https://doi.org/10.1177/0265532208097336
Jang, E. E., Dunlop, M., Park, G., & Van Der Boom, E. H. (2015). How do young students with different profiles of reading skill mastery, perceived ability, and goal orientation respond to holistic diagnostic feedback. Language Testing, 32(3), 359–383. https://doi.org/10.1177/0265532215570924
Jiang, L., & Yu, S. (2022). Appropriating automated feedback in L2 writing: Experiences of Chinese EFL student writers. Computer Assisted Language Learning, 35(7), 1329–1353. https://doi.org/10.1080/09588221.2020.1799824
Jin, L., & Cortazzi, M. (1998). Dimensions of dialogue: Large classes in China. International Journal of Educational Research, 29(8), 739–761. https://doi.org/10.1016/S0883-0355(98)00061-5
Jwa, S. (2019). Transfer of knowledge as a mediating tool for learning: Benefits and challenges for ESL writing instruction. Journal of English for Academic Purposes, 39, 109–118. https://doi.org/10.1016/j.jeap.2019.04.003
Kim, M., & Belcher, D. D. (2018). Building genre knowledge in second language writers during study abroad in higher education. Journal of English for Academic Purposes, 35, 56–69. https://doi.org/10.1016/j.jeap.2018.06.006
Leighton, J. P., Gierl, M. J., & Hunka, S. M. (2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41(3), 205–237. https://doi.org/10.1111/j.1745-3984.2004.tb01163.x
Li, W., Yao, X., & Krstic, M. (2020). Adaptive-gain observer-based stabilization of stochastic strict-feedback systems with sensor uncertainty. Automatica, 120, 109112. https://doi.org/10.1016/j.automatica.2020.109112
Link, S., Mehrzad, M., & Rahimi, M. (2022). Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement. Computer Assisted Language Learning, 35(4), 605–634. https://doi.org/10.1080/09588221.2020.1743323
Liu, C. C., Liu, S. J., Hwang, G. J., Tu, Y. F., Wang, Y., & Wang, N. (2023). Engaging EFL students’ critical thinking tendency and in-depth reflection in technology-based writing contexts: A peer assessment-incorporated automatic evaluation approach. Education and Information Technologies, 1–26. https://doi.org/10.1007/s10639-023-11697-6.
Lu, J., Behbood, V., Hao, P., Zuo, H., Xue, S., & Zhang, G. (2015). Transfer learning using computational intelligence: A survey. Knowledge-Based Systems, 80, 14–23. https://doi.org/10.1016/j.knosys.2015.01.010
Lv, X. (2018). A study on the application of automatic scoring and feedback system in college English writing. International Journal of Emerging Technologies in Learning, 13(03), 188–196. https://doi.org/10.3991/ijet.v13i03.8386
McMillan, J. H., & Schumacher, S. (2010). Research in education: Evidence-based inquiry (7E). Pearson Education Inc..
McNamara, D. S., Crossley, S. A., & Roscoe, R. (2013). Natural language processing in an intelligent writing strategy tutoring system. Behavior Research Methods, 45(2), 499–515. https://doi.org/10.3758/s13428-012-0258-1
Mei, H., & Chen, H. (2022). Assessing students’ translation competence: Integrating China’s standards of English with cognitive diagnostic assessment approaches. Frontiers in Psychology, 1373. https://doi.org/10.3389/fpsyg.2022.872025.
Miles, M. B., Huberman, A. M., & Saldaña, J. (2018). Qualitative data analysis: A methods sourcebook. Sage.
Nash, P., & Shaffer, D. W. (2013). Epistemic trajectories: Mentoring in a game design practicum. Instructional Science, 41, 745–771. https://doi.org/10.1007/s11251-012-9255-0
Pajares, F. (2003). Self-efficacy beliefs, motivation, and achievement in writing: A review of the literature. Reading &Writing Quarterly, 19(2), 139–158. https://doi.org/10.1080/10573560308222
Pajares, F., & Graham, L. (1999). Self-efficacy, motivation constructs, and mathematics performance of entering middle school students. Contemporary Educational Psychology, 24(2), 124–139. https://doi.org/10.1006/ceps.1998.0991
Pajares, F., & Johnson, M. J. (1994). Confidence and competence in writing: The role of self-efficacy, outcome expectancy, and apprehension. Research in the Teaching of English, 313–331.
Pajares, F., & Valiante, G. (1997). Influence of self-efficacy on elementary students' writing. The Journal of Educational Research, 90(6), 353–360. https://doi.org/10.1080/00220671.1997.10544593
Pajares, F., & Valiante, G. (1999). Grade level and gender differences in the writing self-beliefs of middle school students. Contemporary Educational Psychology, 24(4), 390–405. https://doi.org/10.1006/ceps.1998.0995
Pavlenko, A., & Jarvis, S. (2002). Bidirectional transfer. Applied Linguistics, 23(2), 190–214.
Perin, D., Lauterbach, M., Raufman, J., & Kalamkarian, H. S. (2017). Text-based writing of low-skilled postsecondary students: Relation to comprehension, self-efficacy and teacher judgments. Reading and Writing, 30(4), 887–915. https://doi.org/10.1007/s11145-016-9706-0
Perkins, D. N., & Salomon, G. (2012). Knowledge to go: A motivational and dispositional view of transfer. Educational Psychologist, 47(3), 248–258. https://doi.org/10.1080/00461520.2012.693354
Pink, S., Hubbard, P., O’neill, M., & Radley, A. (2010). Walking across disciplines: From ethnography to arts practice. Visual Studies, 25(1), 1–7. https://doi.org/10.1080/14725861003606670
Prat-Sala, M., & Redford, P. (2010). The interplay between motivation, self-efficacy, and approaches to studying. British Journal of Educational Psychology, 80(2), 283–305. https://doi.org/10.1348/000709909X480563
Pratt, S. M., Coleman, J. M., & Dantzler, J. A. (2023). A mixed-methods analysis of fourth-graders’ comprehension and their reported strategies for Reading science texts. Literacy Research and Instruction, 62(1), 16–48. https://doi.org/10.1080/19388071.2022.2039334
Rinnert, C., Kobayashi, H., & Katayama, A. (2015). Argumentation text construction by Japanese as a foreign language writers: A dynamic view of transfer. The Modern Language Journal, 99(2), 213–245. https://doi.org/10.1111/modl.12210
Roscoe, R. D., & McNamara, D. S. (2013). Writing pal: Feasibility of an intelligent writing strategy tutor in the high school classroom. Journal of Educational Psychology, 105(4), 1010. https://doi.org/10.1037/a0032340
Schunk, D., & Pajares, F. (2010). Self-efficacy beliefs. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (3rd ed., pp. 668–672). Elsevier.
Schunk, D. H., & Usher, E. L. (2012). Social cognitive theory and motivation. The Oxford handbook of human motivation.
Shaffer, D. W., Nash, P., & Ruis, A. R. (2015). Technology and the new professionalization of teaching. Teachers College Record, 117(12), 1–30. https://doi.org/10.1177/016146811511701205
Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10.18608/jla.2016.33.3
Shell, A. (1999). Catholicism, controversy and the English literary imagination, 1558–1660. Cambridge University Press.
Shrestha, P. N. (2017). Investigating the learning transfer of genre features and conceptual knowledge from an academic literacy course to business studies: Exploring the potential of dynamic assessment. Journal of English for Academic Purposes, 25, 1–17. https://doi.org/10.1016/j.jeap.2016.10.002
Sorrel, M. A., Olea, J., Abad, F. J., de la Torre, J., Aguado, D., & Lievens, F. (2016). Validity and reliability of situational judgement test scores: A new approach based on cognitive diagnosis models. Organizational Research Methods, 19(3), 506–532. https://doi.org/10.1177/1094428116630065
Spack, R. (1988). Initiating ESL students into the academic discourse community: How far should we go. TESOL Quarterly, 22(1), 29–51. https://doi.org/10.2307/3587060
Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quality of writing. Assessing Writing, 19, 51–65. https://doi.org/10.1016/j.asw.2013.11.007
Storch, N. (2005). Collaborative writing: Product, process, and students’ reflections. Journal of Second Language Writing, 14(3), 153–173. https://doi.org/10.1016/j.jslw.2005.05.002
Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 345–354. https://doi.org/10.1111/j.17453984.1983.tb00212.x.
Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11(3), 287. https://doi.org/10.1037/1082-989X.11.3.287
Teng, L. S., Sun, P. P., & Xu, L. (2018). Conceptualizing writing self-efficacy in English as a foreign language contexts: Scale validation through structural equation modeling. TESOL Quarterly, 52(4), 911–942. https://doi.org/10.1002/tesq.432
Tsao, J. J. (2021). Effects of EFL learners’ L2 writing self-efficacy on engagement with written corrective feedback. The Asia-Pacific Education Researcher, 30(6), 575–584. https://doi.org/10.1007/s40299-021-00591-9
Vojak, C., Kline, S., Cope, B., McCarthey, S., & Kalantzis, M. (2011). New spaces and old places: An analysis of writing assessment software. Computers and Composition, 28(2), 97–111. https://doi.org/10.1016/j.compcom.2011.04.004
Wang, Z. (2020). Computer-assisted EFL writing and evaluations based on artificial intelligence: A case from a college reading and writing course. Library Hi Tech. https://doi.org/10.1108/LHT-05-2020-0113
Wang, S., Yang, Y., Culpepper, S. A., & Douglas, J. A. (2018). Tracking skill acquisition with cognitive diagnosis models: A higher-order, hidden markov model with covariates. Journal of Educational and Behavioral Statistics, 43(1), 57–87. https://doi.org/10.3102/1076998617719727
Wang, Y., Luo, X., Liu, C. C., Tu, Y. F., & Wang, N. (2022). An integrated automatic writing evaluation and SVVR approach to improve students’ EFL writing performance. Sustainability, 14(18), 11586. https://doi.org/10.3390/su141811586
Whyte, M. M., Karolick, D. M., Nielsen, M. C., Elder, G. D., & Hawley, W. T. (1995). Cognitive styles and feedback in computer-assisted instruction. Journal of Educational Computing Research, 12(2), 195–203. https://doi.org/10.2190/M2AV-GEHE-CM9G-J9P7
Wiliam, D. (2011). What is assessment for learning. Studies in Educational Evaluation, 37(1), 3–14. https://doi.org/10.1016/j.stueduc.2011.03.001
Wilson, J., & Roscoe, R. D. (2020). Automated writing evaluation and feedback: Multiple metrics of efficacy. Journal of Educational Computing Research, 58(1), 87–125. https://doi.org/10.1177/0735633119830764
Wilson, J. A., & Soblo, H. (2020). Transfer and transformation in multilingual student writing. Journal of English for Academic Purposes, 44, 100812. https://doi.org/10.1016/j.jeap.2019.100812
Wilson, J., Huang, Y., Palermo, C., Beard, G., & MacArthur, C. A. (2021). Automated feedback and automated scoring in the elementary grades: Usage, attitudes, and associations with writing outcomes in a districtwide implementation of MI write. International Journal of Artificial Intelligence in Education, 31(2), 234–276. https://doi.org/10.1007/s40593-020-00236-w
Wu, W., Huang, J., Han, C., & Zhang, J. (2022). Evaluating peer feedback as a reliable and valid complementary aid to teacher feedback in EFL writing classrooms: A feedback giver perspective. Studies in Educational Evaluation, 73, 101140. https://doi.org/10.1016/j.stueduc.2022.101140
Xu, Y., & Carless, D. (2017). ‘Only true friends could be cruelly honest’: Cognitive scaffolding and social-affective support in teacher feedback literacy. Assessment & Evaluation in Higher Education, 42(7), 1082–1094. https://doi.org/10.1080/02602938.2016.1226759
Yu, S., & Hu, G. (2017). Understanding university students’ peer feedback practices in EFL writing: Insights from a case study. Assessing Writing, 33, 25–35. https://doi.org/10.1016/j.asw.2017.03.004
Yu, S., Jiang, L., & Zhou, N. (2020). Investigating what feedback practices contribute to students’ writing motivation and engagement in Chinese EFL context: A large scale study. Assessing Writing, 44, 100451. https://doi.org/10.1016/j.asw.2020.100451
Yuan, Z. (2021). Interactive intelligent teaching and automatic composition scoring system based on linear regression machine learning algorithm. Journal of Intelligent & Fuzzy Systems, 40(2), 2069–2081. https://doi.org/10.3233/JIFS-189208
Zabihi, R. (2018). The role of cognitive and affective factors in measures of L2 writing. Written Communication, 35(1), 32–57. https://doi.org/10.1177/0741088317735836
Zhan, P., Jiao, H., Liao, D., & Li, F. (2019). A longitudinal higher-order diagnostic classification model. Journal of Educational and Behavioral Statistics, 44(3), 251–281. https://doi.org/10.3102/1076998619827593
Zhang, Z. V. (2020). Engaging with automated writing evaluation (AWE) feedback on L2 writing: Student perceptions and revisions. Assessing Writing, 43, 100439. https://doi.org/10.1016/j.asw.2019.100439
Zimmerman, M. A. (2013). Resiliency theory: A strengths-based approach to research and practice for adolescent health. Health Education & Behavior, 40(4), 381–383. https://doi.org/10.1177/1090198113493782
Zimmerman, B. J., & Bandura, A. (1994). Impact of self-regulatory influences on writing course attainment. American Educational Research Journal, 31(4), 845–862. https://doi.org/10.3102/00028312031004845
Funding
This research received grants from the Major Cultivating Projects of Leading Talents in Philosophy and Social Sciences of Zhejiang Province (NO. 23YJRC13ZD-3YB); Scientific Research Project from Wenzhou University of Technology (NO. KY202220); Research Project of Zhejiang Provincial Education Department (NO. Y202248992).
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Gang Yang, Wei Zhou and Xiaodong Chen. Project administration were performed by Gang Yang, Wei Zhou. Methodology and supervision were performed Gang Yang and Yun-Fang Tu. The first draft of the manuscript was written by Gang Yang, Wei Zhou, Huimin Zhou and Jiawen Li. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval
The ethical requirements for research in this selected university were followed.
Consent to participate
The participants all agreed to take part in this study.
Consent for publication
The publication of this study has been approved by all authors.
Conflicts of interest/competing interests
There is no potential conflict of interest in this study.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yang, G., Zhou, W., Zhou, H. et al. An empirical study of the effects of intelligent cognitive diagnostic feedback strategy on L2 writing performance, epistemic structure, and transferability. Educ Inf Technol 29, 2183–2216 (2024). https://doi.org/10.1007/s10639-023-11905-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-023-11905-3