Abstract
With the rapid development of Internet and explosion of texts, an appropriate way to organize the amount of texts is necessary. Text clustering is of great practical importance for web-learning, which can group similar texts (e.g. documents, textbooks and online notes) to provide users with more valuable information. However, most of existing text clustering algorithms are very sensitive to the parameters needed to be input by users and it is hard to set an appropriate parameter as computers do not know what an appropriate parameter is. Therefore, aiming at this problem, according to the studies of cognitive psychology and our observation, this paper firstly introduces basic level categories and category utility, and then propose a text clustering algorithm to detect basic level categories in texts automatically, which is an non-parametric algorithm. The experimental results show that our algorithm significantly outperforms one basic level concept detection method, k-means and single linkage clustering on different datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aggarwal, C.C., Zhai, C.: A survey of text clustering algorithms. In: Aggarwal, C., Zhai, C. (eds.) Mining Text Data, pp. 77–128. Springer, Boston (2012). doi:10.1007/978-1-4614-3223-4_4
Altman, N.M.: An introduction to kernel and nearest-neighbor nonparametric regression. Am. Stat. 46, 175–185 (1992). Taylor & Francis
Anastasiu, D.C., Tagarelli, A., Karypis, G.: Document clustering: the next frontier. Technical report, University of Minnesota (2013)
Andrews, N.O., Fox, E.A.: Recent developments in document clustering. Technical report, Computer Science, Virginia Tech (2007)
Belohlavek, R., Trnecka, M.: Basic level in formal concept analysis: interesting concepts and psychological ramifications, pp. 1233–1239 (2013)
Cai, Y., Chen, W.-H., Leung, H.-F., Li, Q., Xie, H., Lau, R.Y.K., Min, H., Wang, F.L.: Context-aware ontologies generation with basic level concepts from collaborative tags. Neurocomputing 208, 25–38 (2016)
Dhillon, S.I., Modha, D.S.: Concept decompositions for large sparse text data using clustering. Mach. Learn. 42(1), 143–175 (2001)
Fisher, H.D.: Knowledge acquisition via incremental conceptual clustering. Mach. Learn. 2(2), 139–172 (1987)
Gower, J.C., Ross, G.J.S.: Minimum spanning trees and single linkage cluster analysis. Appl. Stat. 18, 54–64 (1969)
Jones, K.S.: A statistical interpretation of term specificity and its application in retrieval. J. Doc. 28(1), 11–21 (1972)
MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, vol. 1, pp. 281–297 (1967)
Su, J., Lan, M., Tan, C.L., Lu, Y.: Supervised and traditional term weighting methods for automatic text categorization. IEEE Trans. Pattern Anal. Mach. Intell. 31(4), 721–735 (2009)
Manning, C.D., Raghavan, P., Schütze, H., et al.: Introduction to Information Retrieval, vol. 1. Cambridge University Press, Cambridge (2008)
Manning, C.D., Schütze, H., et al.: Foundations of Statistical Natural Language Processing, vol. 999. MIT Press, Cambridge (1999)
Murphy, G.L.: Information, uncertainty and the utility of categories. In: Proceedings of the 7th Annual Conference of Cognitive Science Society, pp. 283–287 (1989)
Murphy, G.L.: The Big Book of Concepts. MIT Press, Cambridge (2004)
Rayson, P., Berridge, D., Francis, B.: Extending the Cochran rule for the comparison of word frequencies between corpora. In: 7th International Conference on Statistical analysis of textual data (JADT 2004), pp. 926–936 (2004)
Rosch, E., Mervis, C.B., Gray, W.D., Johnson, D.M., Boyes-Braem, P.: Basic objects in natural categories. Cognitive Psychol. 8(3), 382–439 (1976)
Leung, H., Cai, Z., Wang, T., Cai, Y., Min, H.: Entropy-based term weighting schemes for text categorization in VSM, pp. 325–332 (2015)
Voorhees, E.M.: Implementing agglomerative hierarchic clustering algorithms for use in document retrieval. Inf. Process. Manag. 22(6), 465–476 (1986)
Willett, P.: Recent trends in hierarchic document clustering: a critical review. Inf. Process. Manag. 24(5), 577–597 (1988)
Acknowledgements
This work is supported by the Fundamental Research Funds for the Central Universities, SCUT (NO. 2017ZD0482015ZM136), Tiptop Scientific and Technical Innovative Youth Talents of Guangdong special support program (No.2015TQ01X633), Science and Technology Planning Project of Guangdong Province, China (No. 2016A030310423), Science and Technology Program of Guangzhou (International Science and Technology Cooperation Program No. 201704030076 and Science and Technology Planning Major Project of Guangdong Province (No. 2015A070711001).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Xu, J. et al. (2017). A Text Clustering Algorithm to Detect Basic Level Categories in Texts. In: Xie, H., Popescu, E., Hancke, G., Fernández Manjón, B. (eds) Advances in Web-Based Learning – ICWL 2017. ICWL 2017. Lecture Notes in Computer Science(), vol 10473. Springer, Cham. https://doi.org/10.1007/978-3-319-66733-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-66733-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66732-4
Online ISBN: 978-3-319-66733-1
eBook Packages: Computer ScienceComputer Science (R0)