Abstract
In view of the traditional classification algorithm, the problem of high feature dimension and data sparseness often occurs when text classification of short texts. This paper proposes a text feature combining neural network language model word2vec and document topic model Latent Dirichlet Allocation (LDA). Represents a matrix model. The matrix model can not only effectively represent the semantic features of the words but also convey the context features and enhance the feature expression ability of the model. The feature matrix was input into the convolutional neural network (CNN) for convolution pooling, and text classification experiments were performed. The experimental results show that the proposed matrix model has better classification effect than the traditional text classification methods based on word2vec and CNN. In the text classification accuracy rate, recall rate and F1 three evaluation indicators increased by 8.4%, 8.9% and 8.6%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Severyn, A., Moschitti, A.: Learning to rank short text pairs with convolutional deep neural networks. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR 2015, pp. 373–382 (2015)
Zhang, Y., Jin, R., Zhou, Z.H.: Understanding bag-of-words model: a statistical framework. Int. J. Mach. Learn. Cybern. 1, 43–52 (2010)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, vol. 26, pp. 3111–3119 (2013)
Zheng, X., Chen, H., Xu, T.: Deep learning for Chinese word segmentation and POS tagging. In: EMNLP, pp. 647–657 (2013)
Xue, B., Fu, C., Shaobin, Z.: A study on sentiment computing and classification of Sina Weibo with Word2vec. In: 2014 IEEE International Congress on Big Data, pp. 358–363 (2014)
Xing, C., Wang, D., Zhang, X., Liu, C.: Document classification with distributions of word vectors. In: Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA 2014 (2014)
Le, Q.V., Mikolov, T.: Distributed representations of sentences and documents. In: International Conference on International Conference on Machine Learning, pp. 1–9 (2014)
Kim, H.K., Kim, H., Cho, S.: Bag-of-concepts: comprehending document representation through clustering words in distributed representation. Neurocomputing 266, 336–352 (2017)
Agarwal, A., Xie, B., Vovsha, I.: Sentiment analysis of Twitter data. In: The Workshop on Languages in Social Media, pp. 30–38. Association for Computational Linguistics (2011)
Yang, F., Li, Z., Zeng, S., Hao, B., Qi, P., Pang, Z.: A novel method for wireless communication signal modulation recognition in smart grid. J. Commun. 11, 813–818 (2016)
Jie, C., Zhiyi, F., Dan, Z., Guannan, Q.: Network traffic classification using feature selection and parameter optimization. Int. J. Appl. Eng. Res. 10, 5663–5679 (2015)
Luong, M., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1412–1421 (2015)
Kim, Y.: Convolutional neural networks for sentence classification. In: The 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751 (2014)
Zhang, Y., Wallace, B.: A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. In: The 8th International Joint Conference on Natural Language Processing, pp. 253–263 (2015)
Mathew, J., Radhakrishnan, D.: An FIR digital filter using one-hot coded residue representation. In: IEEE, pp. 1–4 (2015)
Ming, T., Lei, Z., Xianchun, Z.: Document vector representation based on Word2Vec. Comput. Sci. 43, 214–219 (2016)
Carrera-trejo, V., Sidorov, G., Miranda-jiménez, S., Ibarra, M.M., Martínez, R.C.: Latent Dirichlet allocation complement in the vector space model for multi-label text classification. Int. J. Comb. Optim. Probl. Inform. 6, 7–19 (2015)
Taiyong, G.: A method based on TF-IDF and improved support vector machine research on Chinese text categorization. Comput. Eng. 37, 141–145 (2016)
Yuting, S., Dehua, X.: Research on Chinese text classification based on LDA and SVM. Res. Dev. 2, 18–23 (2016)
Nguyen, T.H., Grishman, R.: Relation extraction: perspective from convolutional neural networks. In: Workshop on Vector Modeling for NLP, pp. 39–48 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, M., Li, T., Huang, P. (2019). Text Classification Research Based on Improved Word2vec and CNN. In: Liu, X., et al. Service-Oriented Computing – ICSOC 2018 Workshops. ICSOC 2018. Lecture Notes in Computer Science(), vol 11434. Springer, Cham. https://doi.org/10.1007/978-3-030-17642-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-17642-6_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-17641-9
Online ISBN: 978-3-030-17642-6
eBook Packages: Computer ScienceComputer Science (R0)