Dynamic EM in Neologism Evolution

Abstract

Research on unsupervised word sense discrimination typically ignores a notable dynamic aspect, whereby the prevalence of a word sense varies over time, to the point that a given word (such as ’tweet’) can acquire a new usage alongside a pre-existing one (such as ’a Twitter post’ alongside ’a bird noise’). This work applies unsupervised methods to text collections within which such neologisms can reasonably be expected to occur. We propose a probabilistic model which conditions words on senses, and senses on times and an EM method to learn the parameters of the model using data from which sense labels have been deleted. This is contrasted with a static model with no time dependency. We show qualitatively that the learned and the observed time-dependent sense distributions resemble each other closely, and quantitatively that the learned dynamic model achieves a higher tagging accuracy (82.4%) than the learned static model does (76.1%).