Abstract
In the era of big data, the internet produces vast amounts of data every day, among which text data occupies the main position. It is difficult for manual processing to deal with the increasing growth rate of text data. As basis of most natural language processing (NLP) tasks, text representation aims to transform text into a vector that can be processed by computer without losing the original important semantic information. It has become an important research direction in the field of NLP that effectively organize, manage and quickly use the complex text information to extract useful semantics from it. Therefore, a text feature representation model based on convolutional neural network (CNN) and variational auto encoder (VAE) is proposed to extract the text features and apply the obtained text feature representation to text classification scene. CNN is used to extract local features and VAE makes the extracted features more consistent with Gaussian distribution. The proposed method has best performance compared with w2v-avg and CNN-AE in k-nearest neighbor (KNN), random forest (RF) and support vector machine (SVM) classification algorithms.
This work was supported in part by National Natural Science Foundation of China (No. 61877010, 11501114), Natural Science Foundation of Fujian Province, China (2019J01243).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ceylan, A.M., Ayta, V.: Concolutional auto encoders for sentence representation generation. Turkish J. Electrical Eng. Comput. Sci. 1135(28) (2020). https://doi.org/10.3906/elk-1907-13
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(1), 2493–2537 (2011). https://doi.org/10.1016/j.chemolab.2011.03.009
Dhingra, B., Zhou, Z., Fitzpatrick, D., Muehl, M., Cohen, W.W.: Tweet2vec: Character-based distributed representations for social media. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (2016)
Guo, C., Guo, W., Chen, C.H., Wang, X., Liu, G.: The air quality prediction based on a convolutional LSTM network. In: Web Information Systems and Applications (2019). https://doi.org/10.1007/978-3-030-30952-7_12
Hershey, J.R., Olsen, P.A.: Approximating the Kullback Leibler divergence between Gaussian mixture models. In: IEEE International Conference on Acoustics (2007). https://doi.org/10.1109/ICASSP.2007.366913
Hill, F., Cho, K., Korhonen, A.: Learning distributed representations of sentences from unlabelled data. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2016). https://doi.org/10.18653/v1/N16-1162
Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Proceedings of Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems (2015)
Ji, S., Yun, H., Yanardag, P., Matsushima, S., Vishwanathan, S.V.N.: Wordrank: Learning word embeddings via robust ranking. In: Computer Science (2015)
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural networkfor modelling sentences. Eprint Arxiv 1 (2014).https://doi.org/10.3115/v1/P14-1062
Kofler, C., Larson, M., Hanjalic, A.: User intent in multimedia search: A survey of the state of the art and future challenges. ACM Comput. Surv. 49(2), 1–37 (2016). https://doi.org/10.1145/2954930
Le, Q.V., Mikolov, T.: Distributed representations of sentences and documents. In: Proceedings of the International Conference on International Conference on Machine Learning (2014)
Liang, H., Sun, X., Sun, Y., Gao, Y.: Text feature extraction based on deep learning: A review. EURASIP J. Wirel. Commun. Netw. 2017(1), 1–12 (2017). https://doi.org/10.1186/s13638-017-0993-1
Liu, G.Z.: Semantic vector space model: Implementation and evaluation. J. Assoc. Inf. Sci. Technol. 48(5), 395–417 (2010). https://doi.org/10.1002/(SICI)1097-4571(199705)48:53.0.CO;2-Q
Liu, N., et. al.: Text representation: From vector to tensor. In: IEEE International Conference on Data Mining (2005). https://doi.org/10.1109/ICDM.2005.144
Mikolov, T.: Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst. 26, 3111–3119 (2013)
Weeds, J., Weir, D.: Co-occurrence retrieval: A flexible framework for lexical distributional similarity. Comput. Lingust. 31(4), 439–475 (2005). https://doi.org/10.1162/089120105775299122
Yin, W., Schutze, H.: Convolutional neural network for paraphrase identification. In: Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 901–911 (2015)
Zhang, R., Lee, H., Radev, D.: Dependency sensitive convolutional neural networks for modeling sentences and documents. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2016). https://doi.org/10.18653/v1/N16-1177
Zhang, Y., Jin, R., Zhou, Z.H.: Understanding bag-of-words model: A statistical framework. Int. J. Mach. Learn. Cybernet. 1(1–4), 43–52 (2010). https://doi.org/10.1007/s13042-010-0001-0
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, C., Xie, L., Liu, G., Wang, X. (2020). A Text Representation Model Based on Convolutional Neural Network and Variational Auto Encoder. In: Wang, G., Lin, X., Hendler, J., Song, W., Xu, Z., Liu, G. (eds) Web Information Systems and Applications. WISA 2020. Lecture Notes in Computer Science(), vol 12432. Springer, Cham. https://doi.org/10.1007/978-3-030-60029-7_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-60029-7_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60028-0
Online ISBN: 978-3-030-60029-7
eBook Packages: Computer ScienceComputer Science (R0)
