Tao Lei
ABSTRACT
Sentence classification, which is the foundation of the subsequent text-based processing, plays an important role in the intelligent question answering (IQA). Convolutional neural networks (CNN) as a kind of common architecture of deep learning, has been widely used to the sentence classification and achieved excellent performance in open field. However, the class imbalance problems and fuzzy sentence feature problem are common in IQA. With the aim to get better performance in IQA, this paper proposes a simple and effective method by increasing generalization and the diversity of sentence features based on simple CNN. In proposed method, the professional entities could be replaced by placeholders to improve the performance of generalization. And CNN reads sentence vectors from both forward and reverse directions to increase the diversity of sentence features. The testing results show that our methods can achieve better performance than many other complex CNN models. In addition, we apply our method in practice of IQA, and the results show the method is effective.
- N. Kalchbrenner, E. Grefenstette, and P. Blunsom. A Convolutional Neural Network for Modelling Sentences. ArXiv e-prints, April 2014.Google Scholar
- Yoon Kim. Convolutional neural networks for sentence classification. Eprint Arxiv, pages 1746--1751, 2014.Google ScholarCross Ref
- Wenpeng Yin and Hinrich Schütze. Multichannel variable-size convolution for sentence classification. arXiv preprint arXiv:1603.04513, 2016.Google Scholar
- Lang Zhining, Gu Xiaozhuo, Zhou Quan, and Xu Taizhong. Combining statisticsbased and cnn-based information for sentence classification. In Tools with Artificial Intelligence (ICTAI), 2016 IEEE 28th International Conference on, pages 1012--1018. IEEE, 2016.Google Scholar
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111--3119, 2013. Google ScholarDigital Library
- Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543, 2014.Google ScholarCross Ref
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug):2493--2537, 2011. Google ScholarDigital Library
- Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.Google Scholar
- Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL '05, pages 115--124, Stroudsburg, PA, USA, 2005. Association for Computational Linguistics. Google ScholarDigital Library
- R Socher, A Perelygin, J.Y. Wu, J Chuang, C.D. Manning, A.Y. Ng, and C Potts. Recursive deep models for semantic compositionality over a sentiment treebank, 01 2013.Google Scholar
- Bo Pang and Lillian Lee. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42Nd Annual Meeting on Association for Computational Linguistics, ACL '04, Stroudsburg, PA, USA, 2004. Association for Computational Linguistics. Google ScholarDigital Library
- Xin Li and Dan Roth. Learning question classifiers. In Proceedings of the 19th International Conference on Computational Linguistics - Volume 1, COLING '02, pages 1--7, Stroudsburg, PA, USA, 2002. Association for Computational Linguistics. Google ScholarDigital Library
- Janyce Wiebe, Theresa Wilson, and Claire Cardie. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation, 39(2):165-- 210, May 2005.Google ScholarCross Ref
- Matthew D. Zeiler. Adadelta: an adaptive learning rate method. CoRR, abs/1212.5701, 2012.Google Scholar
- Quoc Le and Tomas Mikolov. Distributed representations of sentences and documents. In International Conference on Machine Learning, pages 1188--1196, 2014. Google ScholarDigital Library
- Sida Wang and Christopher D Manning. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers-Volume 2, pages 90--94. Association for Computational Linguistics, 2012. Google ScholarDigital Library
- Sida Wang and Christopher Manning. Fast dropout training. In international conference on machine learning, pages 118--126, 2013. Google ScholarDigital Library
- Joao Silva, Luísa Coheur, Ana Cristina Mendes, and Andreas Wichert. From symbolic to sub-symbolic information in question classification. Artificial Intelligence Review, 35(2):137--154, 2011. Google ScholarDigital Library
- Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. Supervised learning of universal sentence representations from natural language inference data. arXiv preprint arXiv:1705.02364, 2017.Google Scholar
- Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, et al. Universal sentence encoder. arXiv preprint arXiv:1803.11175, 2018.Google Scholar
Index Terms
- A novel CNN-based method for Question Classification in Intelligent Question Answering
Recommendations
Robust Neighborhood Preserving Low-Rank Sparse CNN Features for Classification
Advances in Multimedia Information Processing – PCM 2018AbstractConvolutional Neural Networks (CNN) has achieved great success in the area of image recognition, but it usually needs sufficient training data. Meanwhile, similar images tend to deliver compact CNN features, so the original CNN features of ...
Intelligent Question Answering Model Based on CN-BiLSTM
CSAI '18: Proceedings of the 2018 2nd International Conference on Computer Science and Artificial IntelligenceIn this paper, we present a new type of intelligent question answering system model based on deep neural network. In the data stream processing stage, the model proposes a multi-channel information combination framework, which performs operations such ...
Effects of Part-of-Speech on Thai Sentence Classification to Wh-Question Categories using Machine Learning Approach
IAIT '20: Proceedings of the 11th International Conference on Advances in Information TechnologyIn the last decade, question classification is a strong signal for answer selection and help to find the structure of question sentences from sentences. For this paper, we evaluated the proposed pre-processing method for classifying the simple sentence ...
Comments