Tweaking NooJ’s Resources to Export Morpheme-Level or Intra-word Annotations

Abstract

NooJ’s [1, 2] export function allows its users to convert NooJ output from a binary file to a text file format. However, at present, the NooJ export function does not fully export intra-word units. My solution to this problem is demonstrated using SANTI-morf [3], a new morphological annotation system for Indonesian, written as a package in NooJ’s Indonesian language module. While the solution is dedicated to Indonesian morphology, I argue that the method I propose can be replicated by other NooJ users facing the same challenge. A new syntactic grammar whose rules capture morphotactic combinations that form Indonesian polymorphemic words is devised. Dictionaries and morphological grammars are modified to allow all morphemes and their associated analytical attributes to be automatically transferred using the new syntactic grammar as single units, like monomorphemic words. Some symbols that are special to NooJ must be replaced by non-standard symbols as these special symbols are not acceptable in syntactic annotations. This experimentation successfully exports more than 99% of word tokens from the test-bed corpus into full morpheme-level annotations. While successful, the concatenation of these morphemes as single units and the use of non-standard symbols confuse morpheme boundaries and annotations, causing the readability of the output to be low. A small program is then written to improve the readability of the output, which can easily be adapted to users’ anticipated needs.