Ownership Stamp Character Recognition System Based on Ancient Character Typeface

Abstract

In the process of digital archive development of Asian ancient books, ownership stamps and the annotations should be retrievable to provide the origin and versions of the books and the important support of book collection culture. The development of ownership stamp image databases, the text in the ownership stamp, and the background description of the owner will not only narrow the gap between non-professionals and seal culture and art, but also enable people to know more about ownership stamps, which are types of cultural heritage. Meanwhile, it also provides a convenient comparison and reference tool for professional scholars. However, the variety of the written languages used on ownership stamps and the various layouts of the texts and patterns create difficulties for developing such a database. Most of the existing ownership stamp databases have manually created character illustrations for the entire content of the ownership stamp image, but usually no information for the location of the characters that appear in the images, which cause difficulties for non-professional scholars to understand the ancient characters and unknown character sculptural information. Through the usage of a font database, we propose an ancient character ownership stamp database retrieval support system, which will enable users to unscramble the characters of ownership stamps.

1 Introduction

As a special information element of ancient books, ownership stamp (also called collectors’ seal) contains a lot of important information of ancient books. Description about the contents and background of the collection is usually appeared in image databases. Such information includes the annotation of the seal texts, dynasty of the owner, name of the owner, seal script type, shape of the seal, etc. There are different functions of seals existed, such as, showing the ownership, proofing the legend, showing the identity of owners, showing the status, commanding the cherishing, clarifying the aspiration, verifying the versions, and making comments. For Asian countries where Chinese characters are used, it is not only essential to understand the full content of an ancient seal, the understanding of every single character in it is an important matter to understand history and culture as well. We propose a notation recognition system for every single character in the ownership stamps, using the clustering algorithm and multi-feature extraction of ancient character typeface images.

2 Related Work

Fujitsu’s seal recognition project [1] extracts Chinese library seals from the image pages of ancient documents, completes the missing stroke structure, and finally recognizes the complete library seal contents based on matching. We found that most of the existing researches are based on manually labeled databases with large amounts of data and focus on the recognition of the seals as whole. Due to the limited availability of labeled data, we propose a retrieval-based notation recognition system by using multi-feature analysis of ancient character typefaces. In the proposed method, single characters are extracted based on density feature clustering, and through the extraction of multiple features of the image and the calculation with the user input image, we recognize characters in the ancient ownership stamp images.

3 Proposed Method

The proposed method is divided into three steps: (1) character segmentation, (2) multi-feature extraction of ancient character typeface images, and (3) Character recognition by ranking the calculation results.

3.1 Character Segmentation

Different from the usual image binarization processing method, to highlight the spatial distribution characteristics of ownership stamps, k-means clustering heuristic is used in this study to do color quantization of the input image. First, we read the RGB value of the original image and set the k value to 2. Then we divide the result of color quantization into black and white values. Comparing to traditional methods, k-means based binarization method can retain more information of glyphs in the original image. The results will be described in the Sect. 4.

Due to the irregularity of the arrangement of characters in ownership stamps, we do not use the usual histogram watershed algorithm to determine and segment the single character. Because the Chinese characters’ form and configuration have particular characteristics, i.e., structural stability in the center of gravity, and the gap between words, we consider the density of its spatial distribution characteristics. Kernel density estimation is used to analyze the pixel distribution characteristics in binary images. As shown in Fig. 1, in the representation of the three-dimensional image, each axis represents the horizontal and vertical coordinates of the image and the predicted values of its density characteristics respectively. The vertical axis of the three-dimensional coordinate diagram is density characteristics, and we can see that the ownership stamp images have definite distribution characteristics in the spatial distribution. Therefore, we chose the Mean Shift Clustering [2] method based on density to find the local maximum density points of pixel clusters as the center of gravity for clustering. Based on the results of cluster analysis, the text range of seal characters is predicted and processed.

Fig. 1.

Character segmentation based on mean shift clustering

3.2 Multi-feature Extraction of Ancient Character Typeface Images

There are many variations in the shape of an ancient character, a slight change in the shape will result in a change in the calculation result of outline, skeleton feature, and corner feature. In the proposed method, we use the Generative Adversarial Network (GAN) to generate new data of character typeface from public seal script of Shirakawa font [3] and “說文解字True Type字型” [4]. The generated character data is trained as the training data of the VGG 19 [5] and pre-trained model is set as feature extractor to extract the output of pooling layer 3. We use Zhang-Suen thinning algorithm [6] to get the skeleton feature of the original typeface image and input both of original typeface image and their skeleton images to the extractor and obtain the pooling layer 3 feature. In addition, we also extract other features as shown in Table 1.

Table 1. Multi-feature extraction

3.3 Character Recognition by Ranking the Calculation Results

With the pooling layer 3 feature, we can get abstract character features, which can suppress the influence in the ranking results which are caused by shape changes of the same character. Using this feature, pruning operation is performed on the ranking. The result of the top 1,000 characters is used for the final ranking calculation. The ranking calculation method is shown in Fig. 2. The recognition target is also extracted by the method in Sect. 3.2 and the ranking is performed by calculating the Euclidean distance between the features stored in the database and target features. For features with different dimensions, we use hausdorff to calculate their distance. [w1, w2, w3, w4, w5] are the weights to be set for improving the search results. The initial setting is [1, 1, 1, 1, 1] and user interaction can be used to obtain the better results.

Fig. 2.

Ranking calculation method experiments

Fifty randomly selected single characters in scanned images are used for the evaluation experiment. Evaluation of the proposed method is performed using the Mean Reciprocal Rank, and the result is 0.0566 when the weights are in initial setting.

In the Segmentation Evaluation Experiment, Results of the Proposed Method Are Shown in Table 2.

Table 2. The segmentation results

4 Conclusion and Future Work

In this paper, we propose a new method for recognizing characters in the scanned images of ownership stamps using ancient character typefaces. We expect to obtain a retrieval support system with good effectiveness after optimizing the proposed method through parameter adjustments.