Abstract
Short text clustering is beneficial in many applications such as articles recommendations, user clustering and event exploration. Recent works of short text clustering boost the clustering results by improving the representation of short text with deep neural networks, such as CNN and autoencoder. However, existing short text deep clustering methods ignore the structure information of short texts. In this paper, we present a GCN-based clustering method for short text clustering, named as Deep Structured Clustering (DSC) method, to explore the relationships among short texts for representation learning. We first construct a \({\boldsymbol{k}}\)-nn graph to capture the relationships among the short texts, and then jointly learn the short text representations and perform clustering with a dual self-supervised learning module. The experimental results demonstrate the superiority of our proposed method, and the ablation experimental results verify the effectiveness of the modules in our proposed method.
The primary author of this work is a registered student at the time of submission.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bouras, C., Tsogkas, V.: Improving news articles recommendations via user clustering. Int. J. Mach. Learn. Cybern. 8(1), 223–237 (2014). https://doi.org/10.1007/s13042-014-0316-3
Liang, S., Yilmaz, E., Kanoulas, E.: Collaboratively tracking interests for user clustering in streams of short texts. IEEE Trans. Knowl. Data Eng. 31(2), 257–272 (2019)
Feng, W., et al.: Streamcube: hierarchical spatio-temporal hashtag clustering for event exploration over the twitter stream. In: 2015 IEEE 31st International Conference on Data Engineering, pp. 1561–1572 (2015)
Xu, J., et al.: Self-taught convolutional neural networks for short text clustering. Neural Netw. 88, 22–31 (2017)
Xu, J., et al.: Short text clustering via convolutional neural networks. In: Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, Denver, Colorado, pp. 62–69 (2015)
Hadifar, A., Sterckx, L., Demeester, T., Develder, C.: A self-training approach for short text clustering. In: Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), Florence, Italy, pp. 194–199 (2019)
Zhang, D., et al.: Supporting clustering with contrastive learning. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5419–5430 (2021)
Zhang, W., Dong, C., Yin, J., Wang, J.: Attentive representation learning with adversarial training for short text clustering. IEEE Trans. Knowl. Data Eng. (2021). https://doi.org/10.1109/TKDE.2021.3052244. Date of Publication: 18 January 2021
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (2019)
Liu, X., Luo, Z., Huang, H.: Jointly multiple events extraction via attention-based graph information aggregation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 1247–1256 (2018)
Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 7370–7377 (2019)
Bo, D., Wang, X., Shi, C., Zhu, M., Lu, E., Cui, P.: Structural deep clustering network. In: Proceedings of The Web Conference 2020. WWW 2020, pp. 1400–1410. Association for Computing Machinery, New York (2020)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, vol. 26 (2013)
Arora, S., Liang, Y., Ma, T.: A simple but tough-to-beat baseline for sentence embeddings. In: 5th International Conference on Learning Representations, ICLR 2017 (2017)
Kipf, T.N., Welling, M.: Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016)
Wang, C., Pan, S., Hu, R., Long, G., Jiang, J., Zhang, C.: Attributed graph clustering: a deep attentional embedding approach. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp. 3670–3676 (2019)
Nie, F., Wang, X., Jordan, M., Huang, H.: The constrained laplacian rank algorithm for graph-based clustering. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 1969–1976 (2016)
Ng, A., et al.: Sparse autoencoder. CS294A Lect. Notes 72(2011), 1–19 (2011)
Phan, X.-H., Nguyen, L.-M., Horiguchi, S.: Learning to classify short and sparse text & web with hidden topics from large-scale data collections. In: Proceedings of the 17th International Conference on World Wide Web, pp. 91–100 (2008)
Kiros, R., et al.: Skip-thought vectors. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11), 2579–2605 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wu, J., Chen, X., Cai, S., Li, Y., Wu, H. (2022). Deep Structured Clustering of Short Text. In: Liao, X., et al. Big Data. BigData 2021. Communications in Computer and Information Science, vol 1496. Springer, Singapore. https://doi.org/10.1007/978-981-16-9709-8_21
Download citation
DOI: https://doi.org/10.1007/978-981-16-9709-8_21
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-9708-1
Online ISBN: 978-981-16-9709-8
eBook Packages: Computer ScienceComputer Science (R0)
