Abstract
It is an important task to grade answers on specialty subjective questions, which is helpful for the supervision of human review and improving the efficiency and quality of review process. Since this grading process should be performed at the same time with human review, there are only a few samples available for each question that can be provided by specialty experts before review process. We investigate the problem of grading Chinese answers on specialty subjective questions with a reference answer in this paper by proposing a grading model that combines two Bi-LSTM networks with attention mechanism. The first part is a sequence to sequence Bi-LSTM network that adopts the pre-trained word embeddings as input. Since there is no embedding for some specialty words, we instead use the fine-grained word embeddings. After the max-pooling on each sentence, we adopt the mutual attention mechanism to learn the matching degree on specialty knowledge between each pair of sentences of answer and reference. Then we adopt another Bi-LSTM with max-pooling to have an overall vector. By concatenating these two vectors from answer and reference, a multilayer perceptron is adopted to predicate the scores. We adopt the real datasets on a national specialty examination to thoroughly verify the model performance against different amount of training data, network structures, pooling strategies and attention mechanisms. The experimental results show the effectiveness of our method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Burstein, J.: The E-rater® scoring engine: automated essay scoring with natural language processing. Shermis, M.D., Burstein, J.C. (eds.), pp. 113–121 (2003)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27, Annual Conference on Neural Information Processing Systems, pp. 3104–3112. MIT Press, Montreal (2014)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751. ACL, Doha (2014)
Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems 28, Annual Conference on Neural Information Processing Systems, pp. 649–657. ACL, Montreal (2015)
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 655–665. ACL, Baltimore (2014)
Schwenk, H., Barrault, L., Conneau, A., LeCun, Y.: Very deep convolutional networks for text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pp. 1107–1116. ACL, Valencia (2017)
Johnson, R., Zhang, T.: Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 562–570. ACL, Vancouver (2017)
Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432. ACL, Lisbon (2015)
Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 2267–2273. AAAI, Austin (2015)
Shi, Y., Yao, K., Tian, L., Jiang, D.: Deep LSTM based feature mapping for query classification. In: The 2016 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 1501–1511. NAACL, San Diego (2016)
Xiao, Y., Cho, K.: Efficient character-level text classification by combining convolution and recurrent layers. arXiv:1602.00367 (2016)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations. San Diego (2015)
Tan, M., Xiang, B., Zhou, B.: LSTM-based deep learning models for non-factoid answer selection. arXiv:1511.04108 (2015)
Chaturvedi, A., Pandit, O.A., Garain, U.: CNN for text-based multiple choice question answering. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 272–277. ACL, Melbourne (2018)
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A.J., Hovy, E.H.: Hierarchical attention networks for text classification. In: The 2016 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 1480–1489. NAACL, San Diego (2016)
Cui, Y., Liu, T., Chen, Z., Wang, S., Hu, G.: Consensus attention-based neural networks for Chinese reading comprehension. In: 26th International Conference on Computational Linguistics, pp. 1777–1786. ACM, Osaka (2016)
Cui, Y., Chen, Z., Wei, S., Wang, S., Liu, T., Hu, G.: Attention-over-attention neural networks for reading comprehension. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 593–602. ACL, Vancouver (2017)
Li, S., Zhao, Z., Hu, R., Li, W., Liu, T., Du, X.: Analogical reasoning on Chinese morphological and semantic relations. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 138–143. ACL, Melbourne (2018)
Acknowledgments
This work was supported by the National Key R&D Program of China (Grant No. 2018YFC0831401), the National Natural Science Foundation of China (Grant No. 91646119), the Major Project of NSF Shandong Province (Grant No. ZR2018ZB0420), and the Key Research and Development Program of Shandong province (Grant No. 2017GGX10114). The scientific calculations in this paper have been done on the HPC Cloud Platform of Shandong University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, D., Liu, T., Pan, W., Liu, X., Sun, Y., Yuan, F. (2019). Grading Chinese Answers on Specialty Subjective Questions. In: Sun, Y., Lu, T., Yu, Z., Fan, H., Gao, L. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2019. Communications in Computer and Information Science, vol 1042. Springer, Singapore. https://doi.org/10.1007/978-981-15-1377-0_52
Download citation
DOI: https://doi.org/10.1007/978-981-15-1377-0_52
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1376-3
Online ISBN: 978-981-15-1377-0
eBook Packages: Computer ScienceComputer Science (R0)
