Automated Detection of Morphemes Using Distributional Measurements

Abstract

To simply take the distribution of linguistic elements as a basis for analysis was the methodological prime of researchers of the so-called “American Structuralism”. This paper deals with the detection of morphemes from a large corpus of German by simply applying a distributional procedure of counting the number of potential successors of a given sequence of letters of a word, a method reminiscent of proposals by Harris, Shannon and others. Morphemes can be heuristically read off by an increase in the potential successor count. Three different methods of identifying morpheme breaks are discussed and a proposal for improvement of the method by transforming graphemic to partial phonemic representation is put forward.

A. Fenk pointed out to me that the method described does not strictly speaking use an “information theoretical measurement” as the original title suggested. I agree to this appraisal and accordingly replaced the term with “distributional measurements” which — ultimately for historical reasons — might be more appropriate. Thanks to Gustav Vella for painstaking corrections of my “Enklisch”.