Unsupervised word sense disambiguation for Korean through the acyclic weighted digraph using corpus and dictionary

Abstract

Word sense disambiguation (WSD) is meant to assign the most appropriate sense to a polysemous word according to its context. We present a method for automatic WSD using only two resources: a raw text corpus and a machine-readable dictionary (MRD). The system learns the similarity matrix between word pairs from the unlabeled corpus, and it uses the vector representations of sense definitions from MRD, which are derived based on the similarity matrix. In order to disambiguate all occurrences of polysemous words in a sentence, the system separately constructs the acyclic weighted digraph (AWD) for every occurrence of polysemous words in a sentence. The AWD is structured based on consideration of the senses of context words which occur with a target word in a sentence. After building the AWD per each polysemous word, we can search the optimal path of the AWD using the Viterbi algorithm. We assign the most appropriate sense to the target word in sentences with the sense on the optimal path in the AWD. By experiments, our system shows 76.4% accuracy for the semantically ambiguous Korean words.

Introduction

The ambiguity of a word sense can be defined as a natural phenomenon in which a lexical form has more than two senses. WSD is meant to solve this ambiguity of a word sense, that is to say, to assign the appropriate sense to the polysemous word according to its context. It is very difficult to disambiguate polysemous words, but it is very important in the application fields of natural language processing such as machine translation (MT) and information retrieval (IR); for example, we should choose the word of a target language corresponding to a polysemous word in a source context for MT, and the appropriate sense of polysemous words in a query in order to retrieve accurate information from a text for IR. Despite the importance of WSD, the WSD system is rarely used in application fields, for the system learns only a few target words and the performance of the system is very low. In addition to this, it takes long to implement the WSD system because of the need to create sense-labeled examples and construct a thesaurus: this is referred to as a knowledge acquisition bottleneck.

Therefore, we present a way to construct a WSD system, which can be easily implemented by learning all polysemous words at once, while covering all polysemous words which are listed in MRD, and escape from a knowledge acquisition bottleneck, while showing the appropriate accuracy.