Research and Implementation of Buzzword Detection Technology Based on the Dynamic Circulation Corpus

Abstract

Buzzword detection technology is of great significance for reflecting the language life situation, describing the actual state of language usage, and developing language resources. At present, the “Top Ten Annual Buzzwords in Chinese Media” released every year are extracted based on a human-computer cooperation method, which consists of buzzword detection technology and selection by human experts. But the existing buzzword detection technology has the following shortcomings: coarse statistical granularity, slow query, large amount of candidate buzzwords and a high proportion of “non-word” character sequences. In view of those shortcomings, this paper proposes a data storage solution using the InfluxDB time series database based on the daily cycle, two buzzword detection models comprising of logistic curve fitting and abnormal rising segment identification, and utilizing the phrase mining technology for quality scoring and filtering of buzzword candidates. The comparison experiments show that the technology proposed in this paper can achieve better results, significantly reduce the subsequent manual workload, provide more statistical information on buzzword candidates and a more efficient data query method.

Supported by the 2020 Key Project of the 13th Five-Year Plan of the State Language Commission (ZDI135-131), the 2019 Key Project of the 13th Five-Year Plan of the State Language Commission (ZDI135-105), the project of Beijing Advanced Innovation Center for Language Resources (TYZ19005), and the Innovation Fund for Chinese and Foreign Graduate Student (20YCX143).