Elsevier

Pattern Recognition

Volume 36, Issue 4, April 2003, Pages 1045-1060
Pattern Recognition

Classification of forms with handwritten fields by planar hidden Markov models

https://doi.org/10.1016/S0031-3203(02)00123-1Get rights and content

Abstract

In this article, we present a method for modelling physical structure of forms with handwritten fields, by means of pseudo-bidimensional hidden Markov models (PHMMs). This description is then used for automatic classification of types of forms. With the nature of the document, which comprises handwritten fields, position and dimensions of significant rectangles are variable. Moreover, the phenomena of merging and fragmentation, induce an additional variability in the number of rectangles. They characterize the physical structure of a class of forms. Modelling by PHMMs is developed and appears as a suitable tool to solve the problems of the 2D random variability arising from automatic classification of forms.

Introduction

The aims of the automatic analysis of documents are easier communication and storage. When the documents are in paper matter, their conversion in numerical electronic format is made without interpretation in “bitmap” type. In the same way, an unspecified numerical document, of graphic nature, can be converted into “bitmap” format rather easily. So, that is the rough format, which is used as the general representation of the images of forms that we will treat in the continuation. Search in optical character reading led recently to industrial applications: OCR, postal sorting, and automatic reading of the cheques. However, according to our knowledge, the problem of the automatic classification of forms did not find sufficiently good solutions. Search is very active in this field. We can quote Neschen [1] who presented a system for the automatic reading of German banking forms, including a dynamic unit of segmentation, a classifier based on k-nearest neighbour approach, and a semantic correction unit carrying out to search in large data banks. Other authors [2] presented a study of three classifiers for automatic identification of classes of forms. These classifiers are divided into two categories: the first includes the classifier of the k-nearest neighbour and multi-layers perceptron; the second, of structural type, includes a classifier based on trees comparison.

Our work, partially presented in [3], [4], [5], [6], aims at finding a general and reliable method, scientifically founded, which enables the automatic sorting of forms with handwritten fields. Our method rests on the construction of a complex stochastic model, a pseudo-2D hidden Markov model (PHMM), for the automatic recognition of forms with handwritten fields. The physical structure of the form is expressed by the relative layout of the principal rectangles. They contain the zones of texts or images. Horizontal and vertical white bands separate these rectangles. In fact, the principal rectangles are used as a basis for the observations relating to the Markov models. PHMMs were already used for printed writing [7], [8], [9], [10], [11] and handwriting [12], [13], [14], [15] recognition. The most recent work concerning the recognition of the writing comprises phases of manual training. In opposition, our method is entirely automatic, particularly for cutting out the document in bands of super-states.

Section snippets

Planar hidden Markov models

The hidden Markov models (HMM) are currently among the most widespread models in pattern recognition. Firstly introduced within a purely statistical framework [16], [17], then they were essential in speech recognition, afterwards in recognition of the writing. Based on solid theoretical bases, they are characterized by a relative simplicity of the formalisms that they use. Thanks to their flexibility and their elasticity, HMMs were used in many and miscellaneous applications: the modelling of

Architecture

The general structure of the system is described in Fig. 1. The system is conceived in an interactive way: if the type of the form is known, the user can occult the phase of recognition of the form, by introducing directly the corresponding reference. At the time of the phase of learning of interest areas attributes [19], the professor locates the rectangular zones of handwritten data, and the nature and the attributes of the support of the handwritten data (rectangular frame, continuous or

General architecture of the PHMM

Since the treated document is composed of black boxes on white background, we frequently observe sets of identical successive lines; a super-line describes such a unit. In order to compress the representation, a document will be described by a table of super-lines. A super-line is composed of black super-segments. We choose architecture with one vertical principal model; therefore the image of a document must be cut out in homogeneous horizontal bands (which the lines are similar in). A

Construction of the complete average model

From a set of N forms representative of one given class, filled by various writers in the handwritten fields, we can build the complete average model. Each one of these forms is described by a unit of blocks resulting from automatic segmentation of the form. Let us recall that the number of these blocks is not uniform for all forms because of the above-mentioned problems (merging and fragmentation of blocks). It is to be noticed that this model is not the accumulation of all the configurations.

Phase of recognition

After the phase of training is carried out, a PHMM represents each class. The sample to recognize, X, is described by a table of super-lines of observations. An observation is a black segment represented by its length and location parameters. The aim of the processing is double: pairing sample X with the model of the class C, and calculating the conditional probability Prob(X/C). The Bayesian criterion of maximum of probability (maximize Prob(X/C)Prob(C)) enables to make the decision of

Experimentation

The learning base consists of 50 classes. Each class includes 20 forms filled by different scriptwriters. The steps of getting bands of super-states of a model relating to a class is presented in appendix, on a sample of two forms. The recognition was tested on another basis including the 50 same classes (Each class includes 10 specimens filled by different script writers). We asked 250 writers (student and colleagues) to fill the 1000 forms of the learning bank, that is to say 4 forms per

Conclusion

This method for identification of forms depends only on the simplest characteristics related to the physical structure (position and size of the significant rectangular blocks). This choice makes the method flexible and evolutionary, and requires very small memory space to store the data. No sign of reference attached to the document is necessary. The method is independent from the language and the typography used in the form.

At all the learning steps, the construction of the major model, then

Summary

In this article, the authors present a method for modelling physical structure of forms with handwritten fields, by means of pseudo-bidimensional hidden Markov models (PHMMs). This description is then used for automatic classification of types of forms. The present method is based on the detection of main rectangles. They contain zones of texts or images separated by horizontal and vertical white bands. With the nature of the document, which comprises handwritten fields, position and dimensions

About the Author—SAÏD RAMDANE is an Electronic Engineer graduated from the Institute of Batna (Algeria) (1996). He is presently preparing his Ph.D. degree in the University of Le Havre (France), in the field of image processing and pattern recognition.

References (23)

  • M. Neschen, Reconnaissance de Formulaires Manuscrits Basée sur la Quantification Vectorielle Hiérarchique, CNED,...
  • P. Héroux, S. Diana, A. Ribert, E. Trupin, Etude de Méthodes de Classification pour l'Identification Automatique de...
  • S. Kebairi, B. Taconet, A. Zahour, P. Mercy, Détection Automatique du Type de Formulaire Parmi un Ensemble Appris et...
  • S. Kebairi, B. Taconet, A. Zahour, S. Ramdane, A statistical method for an automatic detection of form types,...
  • S. Ramdane, B. Taconet, A. Zahour, S. Kebairi, Apprentissage et Reconnaissance Automatique de Types de Formulaires par...
  • S. Ramdane, B. Taconet, A. Zahour, A. Faure, Modélisation Pseudo Bidimensionnelle pour la Reconnaissance Automatique de...
  • O.E. Agazzi, S. Kuo, E. Levin, R. Pieraccini, Connected and degraded text recognition using planar hidden Markov...
  • N. Ben Amara, A. Belaı̈d, Printed PAW recognition based on planar hidden Markov models, ICPR, 1996, pp....
  • N. Ben Amara, A. Belaı̈d, N. Ellouze, Modélisation Pseudo Bidimensionnelle pour la Reconnaissance de Chaı̂nes de...
  • N. Ben Amara, Utilisation des Modèles de Markov Cachés Planaires en Reconnaissance de l'Ecriture Arabe Imprimée,...
  • S. Kuo et al.

    Keyword spotting in poorly printed documents using pseudo 2D hidden Markov models

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1994)
  • Cited by (0)

    About the Author—SAÏD RAMDANE is an Electronic Engineer graduated from the Institute of Batna (Algeria) (1996). He is presently preparing his Ph.D. degree in the University of Le Havre (France), in the field of image processing and pattern recognition.

    About the Author—BRUNO TACONET obtained his Ph.D. degree in 1978 from the University of Nancy (France). He is a professor of the Informatics Department of the University of Le Havre (France). His fields of interest are document analysis, image processing and pattern recognition.

    About the Author—ABDERRAZAK ZAHOUR obtained his Ph.D. degree in 1990 from the University of Le Havre (France). He is an assistant professor of the Electrical Engineering Department of the University of Le Havre. His fields of interest are document analysis, handwriting recognition and image processing.

    View full text