Abstract
This paper focuses on providing detailed and specific feedback for Chinese writing learners, which is challenging due to the uncertainty of the feedback space. 30 common tags are identified based on clustering 363k comment phrases. By predicting the corresponding tags for an input, learners can gain insights on how to improve their work. However, the various tag types and non-exhaustive annotation pose challenges for model training. To address this, we propose a soft-label-driven approach to construct a more accurate relationship between samples and the tag space. We use a relevance matrix between different tags to adjust the positive label’s confidence across the entire tag space. Additionally, we propose a tag-aware regression-based ranking model that further improves performance. Our experiments demonstrate that the proposed soft-label approach performs better than a hard-label approach, and the proposed ranking model enhances performance. Our data and code are available at: https://github.com/Zhe0311/DetailedFeedback2ChineseWritingLearners.
Y. Cai—Work was done when interning at Microsoft Research Asia.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bekker, J., Davis, J.: Learning from positive and unlabeled data: a survey. Mach. Learn. 109, 719–760 (2020)
Bhatia, K., Jain, H., Kar, P., Varma, M., Jain, P.: Sparse local embeddings for extreme multi-label classification. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)
Cao, N., Zhang, T., Shi, X., Jin, H.: Posistive-unlabeled learning via optimal transport and margin distribution
Chang, W.C., Yu, H.F., Zhong, K., Yang, Y., Dhillon, I.S.: Taming pretrained transformers for extreme multi-label text classification. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3163–3171 (2020)
Das Gupta, M., Srinivasa, S., Antony, M., et al.: Kl divergence based agglomerative clustering for automated vitiligo grading. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2700–2709 (2015)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Dong, F., Zhang, Y., Yang, J.: Attention-based recurrent convolutional neural network for automatic essay scoring. In: Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pp. 153–162 (2017)
Garg, S., Wu, Y., Smola, A.J., Balakrishnan, S., Lipton, Z.: Mixture proportion estimation and PU learning: a modern approach. Adv. Neural. Inf. Process. Syst. 34, 8532–8544 (2021)
Goldberger, J., Gordon, S., Greenspan, H., et al.: An efficient image similarity measure based on approximations of KL-divergence between two gaussian mixtures. In: ICCV, vol. 3, pp. 487–493 (2003)
Gong, J., et al.: IFlyEA: a Chinese essay assessment system with automated rating, review generation, and recommendation. In: Joint Conference on Natural Language Processing: System Demonstrations, pp. 240–248. Association for Computational Linguistics, Online, August 2021. https://doi.org/10.18653/v1/2021.acl-demo.29, https://aclanthology.org/2021.acl-demo.29
Hussein, M.A., Hassan, H., Nassef, M.: Automated language essay scoring systems: a literature review. PeerJ Comput. Sci. 5, e208 (2019)
Jain, H., Prabhu, Y., Varma, M.: Extreme multi-label loss functions for recommendation, tagging, ranking & other missing label applications. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 935–944 (2016)
Ke, Z., Ng, V.: Automated essay scoring: a survey of the state of the art. In: IJCAI, vol. 19, pp. 6300–6308 (2019)
Kiryo, R., Niu, G., Du Plessis, M.C., Sugiyama, M.: Positive-unlabeled learning with non-negative risk estimator. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
OpenAI: ChatGPT (2022). https://openai.com/blog/chatgpt
Ponnoprat, D.: Differential privacy of Dirichlet posterior sampling. CoRR abs/2110.01984 (2021). https://arxiv.org/abs/2110.01984
Prabhu, Y., Kag, A., Harsola, S., Agrawal, R., Varma, M.: Parabel: partitioned label trees for extreme classification with application to dynamic search advertising. In: Proceedings of the 2018 World Wide Web Conference, pp. 993–1002 (2018)
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training (2018)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
Ramesh, D., Sanampudi, S.K.: An automated essay scoring systems: a systematic literature review. Artif. Intell. Rev. 55(3), 2495–2527 (2022)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, Y., Wei, Z., Zhou, Y., Huang, X.J.: Automatic essay scoring incorporating rating schema via reinforcement learning. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 791–797 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cai, Y. et al. (2023). Enhancing Detailed Feedback to Chinese Writing Learners Using a Soft-Label Driven Approach and Tag-Aware Ranking Model. In: Liu, F., Duan, N., Xu, Q., Hong, Y. (eds) Natural Language Processing and Chinese Computing. NLPCC 2023. Lecture Notes in Computer Science(), vol 14302. Springer, Cham. https://doi.org/10.1007/978-3-031-44693-1_45
Download citation
DOI: https://doi.org/10.1007/978-3-031-44693-1_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44692-4
Online ISBN: 978-3-031-44693-1
eBook Packages: Computer ScienceComputer Science (R0)
