Extracting Characteristic Structures among Words in Semistructured Documents

Abstract

Electronic documents such as SGML/HTML/XML files and LaTeX files have been rapidly increasing, by the rapid progress of network and storage technologies. Many electronic documents have no rigid structure and are called semistructured documents. Since a lot of semistructured documents contain large plain texts, we focus on the structural characteristics among words in semistructured documents. The aim of this paper is to present a text mining technique for semistructured documents. We consider a problem of finding all frequent structured patterns among words in semistructured documents. Let (W ₁, W ₂,..., W _k) be a list of words which are sorted in lexicographical order and let k ≥ 2 be an integer. Firstly, we define a tree-association pattern on (W ₁, W ₂,..., W _k). A tree-association pattern on (W ₁, W ₂,..., W _k) is a sequence 〈t ₁; t ₂;...; t _k-1〉 of labeled rooted trees such that, for i = 1, 2,..., k-1, (1) t _i consists of only one node having the pair of two words W _i and W _i+1 as its label, or (2) t _i is a labeled rooted tree which has just two leaves labeled with W _i and W _i+1, respectively. Next, we present a text mining algorithm for finding all frequent tree-association patterns in semistructured documents. Finally, by reporting experimental results on our algorithm, we show that our algorithm is effective for extracting structural characteristics in semistructured documents.