Abstract
This paper implements a semi-supervised text classification method by integrating Paragraph Distributed Representation (PDR) with Extreme Learning Machine (ELM) training algorithm. The proposed Paragraph Distributed Representation-Extreme Learning Machine hybrid classification approach is named as PDR-ELM. Paragraph Distributed Representation is a recently proposed feature selection method based on neural network language model, while Extreme Learning Machine is well known as its high performance in classification. We propose PDR-ELM hybrid classification approach with the objective to minimize the training time and raise the classification accuracy meanwhile. We conduct experiments on a real research paper datasets crawled from Web of Science (WOS). Results show that the proposed PDR-ELM can achieve an accuracy of 81.01% and a training time of 5.1324 seconds on the datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Harris, Z.S.: Distributional structure. Word (1954)
Eyheramendy, S., Lewis, D.D., Madigan, D.: On the naive bayes model for text categorization (2003)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. The Journal of Machine Learning Research 3, 993–1022 (2003)
Bengio, Y., Ducharme, R., Vincent, P., Janvin, C.: A neural probabilistic language model. The Journal of Machine Learning Research 3, 1137–1155 (2003)
Collobert, R., Weston, J.: A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine learning, pp. 160-167. ACM (2008)
Huang, E.H., Socher, R., Manning, C.D., Ng, A.Y.: Improving word representations via global context and multiple word prototypes. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers, vol. 1, pp. 873-882. Association for Computational Linguistics (2012)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013). arXiv preprint arXiv:1301.3781
Adar, E., Dontcheva, M., Laput, G.: CommandSpace: modeling the relationships between tasks, descriptions and features. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, pp. 167-176. ACM (2014)
Le, Q.V., Mikolov, T.: Distributed Representations of Sentences and Documents ICML (2014)
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: theory and applications. Neurocomputing 70, 489–501 (2006)
Hinton, G.E.: Learning distributed representations of concepts. In: Proceedings of the Eighth Annual Conference of the Cognitive Science Sociey (1986)
Mnih, A., Hinton, G.E.: A scalable hierarchical distributed language model. In: NIPS, pp. 1081-1088 (2009)
Mikolov, T., Karafiát, M., Burget, L., Cernocký, J., Khudanpur, S.: Recurrent neural network based language model. In: INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, pp. 1045-1048 (2010)
Sundermeyer, M., Schlüter, R., Ney, H.: LSTM neural networks for language modeling. In: INTERSPEECH (2012)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111-3119 (2013)
Mikolov, T., Yih, W.-t., Zweig, G.: Linguistic regularities in continuous space word representations. In: HLT-NAACL, pp. 746-751 (2013)
Bebis, G., Georgiopoulos, M.: Feed-forward neural networks. IEEE, Potentials 13, 27–31 (1994)
Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 137–142. Springer, Heidelberg (1998)
Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Processing Letters 9, 293–300 (1999)
Huang, G.-B.: An insight into extreme learning machines: random neurons, random features and kernels. Cognitive Computation 6, 376–390 (2014)
Huang, G.-B., Chen, L., Siew, C.-K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Networks 17, 879–892 (2006)
Kasun, L.L.C., Zhou, H., Huang, G.-B., Vong, C.M.: Representational learning with ELMs for big data. IEEE Intelligent Systems 28, 31–34 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Zeng, L., Li, Z. (2015). Text Classification Based on Paragraph Distributed Representation and Extreme Learning Machine. In: Tan, Y., Shi, Y., Buarque, F., Gelbukh, A., Das, S., Engelbrecht, A. (eds) Advances in Swarm and Computational Intelligence. ICSI 2015. Lecture Notes in Computer Science(), vol 9141. Springer, Cham. https://doi.org/10.1007/978-3-319-20472-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-20472-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20471-0
Online ISBN: 978-3-319-20472-7
eBook Packages: Computer ScienceComputer Science (R0)