Abstract
The application of traditional text clustering methods to short text data is inefficient owing to the high dimensionality and semantic sparseness of such data. Contrastingly, convolutional neural networks can capture the local information between consecutive words in a sentence and extract the semantic features of the text. In this paper, we propose a short text clustering method based on convolutional autoencoders (CAE-STC) that jointly optimizes feature representations and cluster assignments. The proposed method employs a convolutional autoencoder to learn deep text feature representations and preserve the local structure of text generation distribution. By integrating the clustering loss and convolutional autoencoder’s reconstruction loss, a unified loss function is formulated to update the network parameters and cluster centers iteratively, improving the performance of the feature learning and clustering tasks. The results of extensive experiments conducted on three public short text datasets demonstrate that the proposed method outperforms several popular clustering methods in terms of the normalized mutual information and clustering accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jung, H., Lee, B.G.: Research trends in text mining: semantic network and main path analysis of selected journals. Expert Syst. Appl. 162, 1–12 (2020)
Rezaeinia, S.M., Rahmani, R., Ghodsi, A., Veisi, H.: Sentiment analysis based on improved pre-trained word embeddings. Expert Syst. Appl. 117(1), 139–147 (2019)
Chen, J., Gong, Z., Liu, W.: A nonparametric model for online topic discovery with word embeddings. Inf. Sci. 504, 32–47 (2019)
Campos, L.M.D., Fernández-Luna, J.M., Huete, J.F., Redondo-Expósito, L.: Automatic construction of multi-faceted user profiles using text clustering and its application to expert recommendation and filtering problems. Knowl.-Based Syst. 29, 1–18 (2020)
Meht, V., Bawa, S., Singh, J.: Stamantic clustering: combining statistical and semantic features for clustering of large text datasets. Expert Syst. Appl. 174(15), 1–9 (2021)
Wei, T., Lu, Y., Chang, H., Zhou, Q., Bao, X.: A semantic approach for text clustering using WordNet and lexical chains. Expert Syst. Appl. 42(4), 2264–2275 (2015)
Zhang, L., Wang, S., Liu, B.: Deep learning for sentiment analysis: a survey. WIREs Data Mining Knowl. Discov. 8(4), 1–25 (2018)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of Workshop at ICLR, pp. 1–12 (2013)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751. ACL, Doha (2014)
Gharavi, E., Veisi, H., Silwal, R., Gerber, M.S.: Improving discourse representations with node hierarchy attention. Mach. Learn. Appl. 3, 1–7 (2021)
Hu, X., Sun, N., Zhang, C., Chua, T.-S.: Exploiting internal and external semantics for the clustering of short texts using world knowledge. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 919–928. ACM, Hong Kong (2009).
Bouras, C., Tsogkas, V.: A clustering technique for news articles using WordNet. Knowl.-Based Syst. 36, 115–128 (2012)
Chen, X., Zhang, Y., Cao, L., Li, D.: An improved feature selection method for chinese short texts clustering based on HowNet. In: Wong, W.E., Zhu, T. (eds.) Computer Engineering and Networking. LNEE, vol. 277, pp. 635–642. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-01766-2_73
Mizzaro S., Pavan M., Scagnetto I., Valenti M.: Short text categorization exploiting contextual enrichment and external knowledge. In: Proceedings of the First International Workshop on Social Media Retrieval and Analysis, pp. 57–62. ACL, Gold Coast (2014)
Yang, J., Li, Y., Gao, C., Zhang, Y.: Measuring the short text similarity based on semantic and syntactic information. Futur. Gener. Comput. Syst. 114, 169–180 (2021)
Beil, F., Ester, M., Xu, X.: Frequent term-based text clustering. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 436–442. ACM, Edmonton (2002)
Peng, M., Huang, J., Zhu, J., Huang, J., Liu, J.: Mass of short texts clustering and topic extraction based on frequent itemsets. J. Comput. Res. Develop. 52(9), 1941–1953 (2015)
Cekik, R., Uysal, A.K.: A novel filter feature selection method using rough set for short text data. Expert Syst. Appl. 160, 1–15 (2020)
Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543. ACL, Doha (2014)
Xu, J., Wang, P., Tian, G., Xu, B., Zhao, J., Wang, F., Hao, H.: Short text clustering via convolutional neural networks. In: Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pp. 62–69. ACL, Denver (2015)
Xu, J., et al.: Self-taught convolutional neural networks for short text clustering. Neural Netw. 88, 22–31 (2017)
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: Proceedings of the 33rd International Conference on Machine Learning, pp. 1–10. ACM, New York (2016)
Guo, X., Liu, X., Zhu, E., Yin, J.: Deep clustering with convolutional autoencoders. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.-S.M. (eds.) ICONIP 2017, LNCS (LNTCS), vol. 10635, pp. 373–382. Springer, Guangzhou (2017). https://doi.org/10.1007/978-3-319-70096-0_39
Li, F., Qiao, H., Zhang, B.: Discriminatively boosted image clustering with fully convolutional auto-encoders. Pattern Recogn. 83, 161–173 (2018)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Phan, X.-H.., Nguyen, L.-M., Horiguchi, S.: Learning to classify short and sparse text & web with hidden topics from large-scale data collections. In: WWW, pp. 91–100. ACM, Beijing (2008)
Zheng, C.T., Liu, C., Sanwong, H.: Corpus-based topic diffusion for short text clustering. Neurocomputing 275(31), 2444–2458 (2018)
Janani, R., Vijayarani, D.S.: Text document clustering using spectral clustering algorithm with particle swarm optimization. Expert Syst. Appl. 134(15), 192–200 (2019)
Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Constrained K-means clustering with background knowledge. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 577–584. ACM, San Francisco (2001)
Socher, R., Pennington, J., Huang, E.H., Ng, A.Y., Manning, C.D.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 151–161. ACL, Edinburgh (2011)
Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Acknowledgments
This research is supported by the Anhui Provincial Natural Science Foundation of China (No. 2108085MF214), the Key Program in the Youth Elite Support Plan in Universities of Anhui Province (No. gxyqZD2020004) and National Natural Science Foundation of China (No. 61972439).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, L., Du, T., Duan, X., Luo, Y. (2021). Short Text Clustering Using Joint Optimization of Feature Representations and Cluster Assignments. In: Pham, D.N., Theeramunkong, T., Governatori, G., Liu, F. (eds) PRICAI 2021: Trends in Artificial Intelligence. PRICAI 2021. Lecture Notes in Computer Science(), vol 13032. Springer, Cham. https://doi.org/10.1007/978-3-030-89363-7_17
Download citation
DOI: https://doi.org/10.1007/978-3-030-89363-7_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89362-0
Online ISBN: 978-3-030-89363-7
eBook Packages: Computer ScienceComputer Science (R0)