Multilayer cluster neural network for totally unconstrained handwritten numeral recognition

doi:10.1016/0893-6080(95)00020-Z

Neural Networks

Volume 8, Issue 5, 1995, Pages 783-792

https://doi.org/10.1016/0893-6080(95)00020-Z Get rights and content

Abstract

In this paper, we propose a simple multilayer cluster neural network with five independent subnetworks for off-line recognition of totally unconstrained handwritten numerals. We also show that the use of genetic algorithms for avoiding the problem of finding local minima in training the multilayer cluster neural network with gradient descent technique reduces error rates.

In order to verify the performance of the proposed multilayer cluster neural network, experiments with unconstrained handwritten numeral database of Concordia University of Canada, that of Electro-Technical Laboratory of Japan, and that of Electronics and Telecommunications Research Institute of Korea were performed. For the case of determining the initial weights of the multilayer cluster neural network randomly, the error rates were 2.90%, 1.50%, and 0.80%, respectively. And, for the case of determining the initial weights using a genetic algorithm, the error rates were 2.20%, 0.87%, and 0.60%, respectively.

References (33)

AhmedP. et al.
Computer recognition of totally unconstrained handwritten ZIP condes
International Journal of Pattern Recognition and Artificial Intelligence
(1987)
BäckT.
A user's guide to GENEsYs 1.0
(1992)
BelewR.K. et al.
Evolving networks: Using the genetic algorithm with connectionist learning
BeunM.
A flexible method for automatic reading of handwritten numerals
Philips Technical Review
(1973)
BeunM.
A flexible method for automatic reading of handwritten numerals
Philips Technical Review
(1973)
CohenE. et al.
Understanding handwritten text in a structured environment: Determining ZIP codes from addresses
International Journal of Pattern Recognition and Artificial Intelligence
(1991)
DruckerH. et al.
Boosting performance in neural networks
International Journal of Pattern Recognition and Artificial Intelligence
(1993)
DuerrB. et al.
A Combination of Statistical and Syntactical Pattern Recognition Applied to Classification of Unconstrained Handwritten Numerals
Pattern Recognition
(1980)
FukushimaK.
Neocognitron: A hierarchical neural network capable of visual pattern recognition
Neural Networks
(1988)
GaderP.D. et al.
Pipelined systems for recognition of handwritten digit in USPS ZIP codes

HollandJ.H.

Adaptation in natural and artificial systems

(1992)

KitanoH.

Empirical studies on the speed of convergence of neural network training using genetic algorithm

KnerrS. et al.

Handwritten digit recognition by neural networks with single-layer training

IEEE Transactions on Neural Networks

(1992)

KrzyzakA. et al.

Unconstrained handwritten character classification using modified backpropagation model

KuanC.L. et al.

A stroke-based approach to handwritten numeral recognition

LamL. et al.

Structural classification and relaxation matching of totally unconstrained handwritten ZIPcode numbers

Pattern Recognition

(1988)

Cited by (40)

Swarm Based-Artificial Neural System for Human Health Data Classification
2015, Applied Computing in Medicine and Health
Cancer is an important medical disorder, which is not a single disease but a cluster more than 200 different serious medical complications. The accurate prediction in patients with early-stage cancer is of significant importance to reduce the mortality rate of those patients. Therefore, biologically inspired approaches that are motivated by the natural behaviors of swarms are used in this work. They are robust, easy to implement, and has few setting parameters. However, one disadvantage is that they are of slow convergence, which is due to the poor exploration and exploitation processes. To overcome this deficiency of the traditional algorithm, we propose the Global Guided Artificial Bee Colony (GGABC) and Hybrid Guided Artificial Bee Colony (HGABC) algorithms; which employ new hybrid population-based meta-heuristic approaches, simulated by the foraging behaviors of global best and guided honey bees. In this chapter, GGABC and HGABC algorithms are proposed in order to determine whether patients have cancer or not. The simulation results of the proposed approaches were compared with other algorithms including ABC, Guided ABC, and Global ABC. The classification results of cancer by GGABC and HGABC models are highly accurate in contrast to the results given by the benchmarked algorithms.
A classifier for Bangla handwritten numeral recognition
2012, Expert Systems with Applications
This paper presents a novel pattern classification approach – a kernel and Bayesian discriminant based classifier which utilizes the distribution characteristics of the samples in each class. A kernel combined with Bayesian discriminant in the subspace spanned by the eigenvectors which are associated with the smaller eigenvalues in each class is adopted as the classification criterion. To solve the problem of the matrix inverse, the smaller eigenvalues are substituted by a small threshold which is decided by minimizing the training error in a given database. Application of the proposed classifier to the issue of handwritten numeral recognition demonstrates that it is promising in practical applications.
Handwritten digit recognition: Investigation of normalization and feature extraction techniques
2004, Pattern Recognition
Citation Excerpt :
For gradient feature extraction, we use the Sobel operator to compute the x/y components of gradient and the gradient image is decomposed into four orientation planes or eight direction planes. The Sobel operator has been used by other researchers [17,18], and some researchers have also used the Roberts operator [19] and Kirsh operator [28]. The gradient strength and direction can be computed from the vector [gx,gy]T. For character feature extraction, the gradient of every pixel on the normalized image is computed.
The performance evaluation of various techniques is important to select the correct options in developing character recognition systems. In our previous works, we have proposed aspect ratio adaptive normalization (ARAN) and have evaluated the performance of state-of-the-art feature extraction and classification techniques. For this time, we will propose some improved normalization functions and direction feature extraction strategies and will compare their performance with existing techniques. We compare ten normalization functions (seven based on dimensions and three based on moments) and eight feature vectors on three distinct data sources. The normalization functions and feature vectors are combined to produce eighty classification accuracies to each dataset. The comparison of normalization functions shows that moment-based functions outperform the dimension-based ones and the aspect ratio mapping is influential. The comparison of feature vectors shows that the improved feature extraction strategies outperform their baseline counterparts. The gradient feature from gray-scale image mostly yields the best performance and the improved NCFE (normalization-cooperated feature extraction) features also perform well. The combined effects of normalization, feature extraction, and classification have yielded very high accuracies on well-known datasets.
Handwritten digit recognition: Benchmarking of state-of-the-art techniques
2003, Pattern Recognition
Citation Excerpt :
The gradient image is decomposed into four orientation planes or eight direction planes. The gradient operators used in feature extraction of character recognition include the Roberts operator [7,45], the Sobel operator [50,53] and the Kirsh operator [33]. The Kirsh operator is unique in that four gradient components are computed while other operators compute two components.
This paper presents the results of handwritten digit recognition on well-known image databases using state-of-the-art feature extraction and classification techniques. The tested databases are CENPARMI, CEDAR, and MNIST. On the test data set of each database, 80 recognition accuracies are given by combining eight classifiers with ten feature vectors. The features include chaincode feature, gradient feature, profile structure feature, and peripheral direction contributivity. The gradient feature is extracted from either binary image or gray-scale image. The classifiers include the k-nearest neighbor classifier, three neural classifiers, a learning vector quantization classifier, a discriminative learning quadratic discriminant function (DLQDF) classifier, and two support vector classifiers (SVCs). All the classifiers and feature vectors give high recognition accuracies. Relatively, the chaincode feature and the gradient feature show advantage over other features, and the profile structure feature shows efficiency as a complementary feature. The SVC with RBF kernel (SVC-rbf) gives the highest accuracy in most cases but is extremely expensive in storage and computation. Among the non-SV classifiers, the polynomial classifier and DLQDF give the highest accuracies. The results of non-SV classifiers are competitive to the best ones previously reported on the same databases.
Local learning framework for handwritten character recognition
2002, Engineering Applications of Artificial Intelligence
This paper proposes a general local learning framework to effectively alleviate the complexities of classifier design by means of “divide and conquer” principle and ensemble method. The learning framework consists of a quantization layer which uses generalized learning vector quantization (GLVQ) and an ensemble layer which uses multi-layer perceptrons (MLP). The proposed method is tested on public handwritten character data sets, which obtains a promising performance consistently. In contrast to other methods, the proposed method is especially suitable for a large-scale real-world classification problems although it is easily scaled to a small training set while preserving a good performance.
Fusion of multiple handwritten word recognition techniques
2001, Pattern Recognition Letters
Fusion of multiple handwritten word recognition techniques is described. A novel Borda count for fusion based on ranks and confidence values is proposed. Three techniques with two different conventional segmentation algorithms in conjunction with backpropagation and radial basis function neural networks have been used in this research. Development has taken place at the University of Missouri and Griffith University. All experiments were performed on real-world handwritten words taken from the CEDAR benchmark database. The word recognition results are very promising and the highest (91%) among published results for handwritten words.

View all citing articles on Scopus

View full text

Multilayer cluster neural network for totally unconstrained handwritten numeral recognition

Abstract

Computer recognition of totally unconstrained handwritten ZIP condes

International Journal of Pattern Recognition and Artificial Intelligence

A user's guide to GENEsYs 1.0

Evolving networks: Using the genetic algorithm with connectionist learning

A flexible method for automatic reading of handwritten numerals

Philips Technical Review

A flexible method for automatic reading of handwritten numerals

Philips Technical Review

Understanding handwritten text in a structured environment: Determining ZIP codes from addresses

International Journal of Pattern Recognition and Artificial Intelligence

Boosting performance in neural networks

International Journal of Pattern Recognition and Artificial Intelligence

A Combination of Statistical and Syntactical Pattern Recognition Applied to Classification of Unconstrained Handwritten Numerals

Pattern Recognition

Neocognitron: A hierarchical neural network capable of visual pattern recognition

Neural Networks

Pipelined systems for recognition of handwritten digit in USPS ZIP codes

Adaptation in natural and artificial systems

Empirical studies on the speed of convergence of neural network training using genetic algorithm

Handwritten digit recognition by neural networks with single-layer training

IEEE Transactions on Neural Networks

Unconstrained handwritten character classification using modified backpropagation model

A stroke-based approach to handwritten numeral recognition

Structural classification and relaxation matching of totally unconstrained handwritten ZIPcode numbers

Pattern Recognition