Abstract
With the advent of the era of big data, information overload has become a universal problem. How to extract valuable information from the complicated data, and let users quickly understand the main content of these data has become the focus of research. Current topic generation is a way to solve this problem. Some unsupervised topic models generate results for the probability distribution of topics and words, which are less explanatory to the topic, unable to generate subject matter that can be read smoothly. Supervised topic classification methods rely too much on tagged training data and are not universally applicable. This paper proposes a novel topic generation algorithm DTR, which is based on Dbscan and TextRank. By combining text clustering with abstract algorithm, it can automatically generate refined and understandable document topics. At the same time, the similarity analysis method is used to quickly process the document, which narrows the data range of the generated topic and improves the accuracy of the generated topic. The effectiveness of this algorithm is verified by experiments based on a large amount of real data.
This work was supported by State Grid Technical Project (No. 52110418002X).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mu, X., Hao, W., Gang, C., Zhao, S., Jin, D.: Research based on concept maps and hidden Markov model for multi-document summary. In: IEEE International Conference on Broadband Network Multimedia Technology (2011)
Nema, P., Khapra, M.M., Laha, A., Ravindran, B.: Diversity driven attention model for query-based abstractive summarization (2017)
Perotte, A., Bartlett, N., Elhadad, N., Wood, F.: Hierarchically supervised latent Dirichlet allocation. In: International Conference on Neural Information Processing Systems (2011)
Koltcov, S., Koltsova, O., Nikolenko, S.: Latent Dirichlet allocation: stability and applications to studies of user-generated content. In: ACM Conference on Web Science (2014)
Blei, D.M., Mcauliffe, J.D.: Supervised topic models. In: Advances in Neural Information Processing Systems, vol. 3, pp. 327–332 (2010)
Ramage, D., Hall, D., Nallapati, R., Manning, C.D.: Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In: Proceedings of EMNLP, vol. 1, pp. 248–256 (2009)
Ma, S., Sun, X., Lin, J., Wang, H.: Autoencoder as assistant supervisor: Improving text representation for Chinese social media text summarization (2018)
Shanshan, Y., Jindian, S., Pengfei, L.: Improved TextRank-based method for automatic summarization (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Cui, Y., Liu, D., Li, Q., Qiu, Z., Yang, X. (2020). DTR: A Novel Topic Generate Algorithm Based on Dbscan and TextRank. In: Liu, Y., Wang, L., Zhao, L., Yu, Z. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2019. Advances in Intelligent Systems and Computing, vol 1075. Springer, Cham. https://doi.org/10.1007/978-3-030-32591-6_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-32591-6_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32590-9
Online ISBN: 978-3-030-32591-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)