An Ontology for Generalized Disease Incidence Detection on Twitter

Abstract

In this paper, we present an ontology of disease related concepts that is designated for detection of disease incidence in tweets. Unlike previous key word based systems and topic modeling approaches, our ontological approach allows us to apply more stringent criteria for determining which messages are relevant such as spatial and temporal characteristics whilst giving a stronger guarantee that the resulting models will perform well on new data that may be lexically divergent. We achieve this by training supervised learners on concepts rather than individual words. Effectively, we map every possible word to a fixed length lexicon thereby eliminating lexical divergence between training data and new data. For training we use a dataset containing mentions of influenza, common cold and Listeria and use the learned models to classify datasets containing mentions of an arbitrary selection of other diseases. We show that our ontological approach results in models whose performance is not only good but also stable on lexically divergent data versus a word-level lookup unigram, bag of words baseline approach. We also show that word vectors can be learned directly from our concepts to achieve even better results.