Handwritten Chinese text line segmentation by clustering with distance metric learning

Abstract

Separating text lines in unconstrained handwritten documents remains a challenge because the handwritten text lines are often un-uniformly skewed and curved, and the space between lines is not obvious. In this paper, we propose a novel text line segmentation algorithm based on minimal spanning tree (MST) clustering with distance metric learning. Given a distance metric, the connected components (CCs) of document image are grouped into a tree structure, from which text lines are extracted by dynamically cutting the edges using a new hypervolume reduction criterion and a straightness measure. By learning the distance metric in supervised learning on a dataset of pairs of CCs, the proposed algorithm is made robust to handle various documents with multi-skewed and curved text lines. In experiments on a database with 803 unconstrained handwritten Chinese document images containing a total of 8,169 lines, the proposed algorithm achieved a correct rate 98.02% of line detection, and compared favorably to other competitive algorithms.

Introduction

Text line segmentation from document images is one of the major problems in document image analysis. It provides crucial information for the tasks of text block segmentation, character segmentation and recognition, and text string recognition. Whereas the difficulty of machine-printed document analysis mainly lies in the complex layout structure and degraded image quality, handwritten document analysis is difficult mainly due to the irregularity of layout and character shapes originated from the variability of writing styles. For unconstrained handwritten documents, text line segmentation and character segmentation-recognition are not solved though enormous efforts have been devoted to them and great advances have been made.

Text line segmentation of handwritten documents is much more difficult than that of printed documents. Unlike that printed documents have approximately straight and parallel text lines, the lines in handwritten documents are often un-uniformly skewed and curved. Moreover, the spaces between handwritten text lines are often not obvious compared to the spaces between within-line characters, and some text lines may interfere with each other. Therefore, many text line detection techniques, such as projection analysis [1], [2], [3], [4], [5], [6], [7] and K-nearest neighbor connected components (CCs) grouping [12], [13], [14], are not able to segment handwritten text lines successfully. Fig. 1 shows an example of unconstrained handwritten Chinese document with segmentation results by the X–Y cut algorithm [1], the stroke skew correction algorithm [6], the Docstrum algorithm [12] and the piece-wise projection algorithm [5]. In this case, we can see that the X–Y cut algorithm and the stroke skew correction algorithm succeed in detecting the text lines, but fail to locate the boundaries of text lines. The Docstrum algorithm can locate the boundaries of text lines very well, but fails to detect some lines (the first and fourth lines in Fig. 1(c)) correctly because of the anomalous size of characters. Although the piece-wise projection algorithm can overcome the aforementioned errors, it fails to segment some small-size CCs (the first and eighth lines in Fig. 1(d)).

Many efforts have been devoted to the difficult problem of handwritten text line segmentation [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28]. The methods can be roughly categorized into three classes: top-down, bottom-up, and hybrid. Top-down methods partition the document image recursively into text regions, text lines, and words/characters with the assumption of straight lines. Bottom-up methods group small units of image (pixels, CCs, characters, words, etc.) into text lines and then text regions. Bottom-up grouping can be viewed as a clustering process, which aggregates image components according to proximity and does not rely on the assumption of straight lines. Hybrid methods combine bottom-up grouping and top-down partitioning in different ways. All the three approaches have their disadvantages. Top-down methods do not perform well on curved and overlapping text lines. The performance of bottom-up grouping relies on some heuristic rules or artificial parameters, such as the between-component distance metric for clustering. On the other hand, hybrid methods are complicated in computation, and the design of a robust combination scheme is non-trivial.

In this paper, we propose an effective bottom-up method for text line segmentation in unconstrained handwritten Chinese documents. Our approach is based on minimal spanning tree (MST) clustering of CCs and the distance metric between CCs is designed by supervised learning. The number of clusters, namely the number of text lines, is automatically decided by a new hypervolume reduction criterion. Except for some empirical parameters in pre-processing of CCs and in post-processing of text lines, the clustering algorithm itself has no artificial parameters. The experimental comparison of clustering with metric learning with that of artificially designed metric shows that supervised metric learning improves largely the accuracy of text line segmentation. The proposed method was also compared with other state-of-the-art methods in experiments on a large database of handwritten Chinese documents and its superiority was demonstrated. By customizing the between-component features and training with documents of specific languages, we suggest that the proposed method is also applicable to the documents of other languages.

The rest of this paper is organized as follows. In Section 2, we give a brief review of the related works; An overall description of our clustering-based text line segmentation method is given in Section 3, and the distance metric learning scheme is elaborated in Section 4. In Section 5, we present the hypervolume reduction criterion and the straightness measure for text line grouping. Experimental results are presented in Section 6 and concluding remarks are given in Section 7.