Abstract
Due to short length and limited content, short text representation has the problem of high-dimension and high-sparsity. For the purpose of achieving the goal of reducing the dimension and eliminate the sparseness while preserve the semantics of the information in the text to be represented, a method of short text mapping based on fast clustering using minimum spanning trees is proposed. First, we remove the irrelevant terms, then a clustering method based on minimum spanning tree is adopted to identify the relevant term set and remove the redundant terms to get the short text mapping space. Finally, a matrix mapping method is designed to represent the original short text on a highly correlated and non-redundant short text mapping space. The proposed method not only has low time complexity but also produces higher quality short text mapping space. The experiments prove that our method is feasible and effective.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yong, Z., Li, Y., Xia, S.: An improved KNN text classification algorithm based on clustering. J. Comput. 4(3), 230–237 (2009)
Cai, Y., et al.: Semi-supervised short text categorization based on attribute selection. J. Comput. Appl. 30(4), 1015–1018 (2010)
Li, P., Wang, H., Zhu, K.Q., et al.: A large probabilistic semantic network based approach to compute term similarity. IEEE Trans. Knowl. Data Eng. 27(10), 2604–2617 (2015)
Kumar, S., Rengarajan, P., Annie, A.X.: Using wikipedia category network to generate topic trees. In: AAAI 2017, pp. 4951–4952 (2017)
Piao, G.Y., Breslin, J.G.: User modeling on Twitter with WordNet synsets and DBpedia concepts for personalized recommendations. In: CIKM 2016, pp. 2057–2060 (2016)
Sun, A.: Short text classification using very few words. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1145–1146. ACM (2012)
Usama, M.F., Irani, K.B.: Multi-interval discretization of continuous valued attributes for classification learning. In: Proceedings of 13th International Joint Conference on AI, pp. 1022–1027 (1993)
John, G.H., Kohavi, R., Pfleger, K.: Irrelevant features and the subset selection problem. In: The Proceedings of the Eleventh International Conference on Machine Learning, pp. 121–129 (1994)
Song, Q., Ni, J., Wang, G.: A fast clustering-based feature subset selection algorithm for high-dimensional data. IEEE Trans. Knowl. Data Eng. 25(1), 1–14 (2013)
Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1979)
Prim, R.C.: Shortest connection networks and some generalizations. Bell Syst. Tech. J. 36, 1389–1401 (1957)
Gao, L., Zhou, S., Guan, J.: Effectively classifying short texts by structured sparse representation with dictionary filtering. Inf. Sci. 323, 130–142 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, P. (2019). Short Text Mapping Based on Fast Clustering Using Minimum Spanning Trees. In: Huang, DS., Huang, ZK., Hussain, A. (eds) Intelligent Computing Methodologies. ICIC 2019. Lecture Notes in Computer Science(), vol 11645. Springer, Cham. https://doi.org/10.1007/978-3-030-26766-7_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-26766-7_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-26765-0
Online ISBN: 978-3-030-26766-7
eBook Packages: Computer ScienceComputer Science (R0)