A new method for information update in supervised neural structures
Introduction
Due to the increasing number of available data that must be analyzed and classified in modern computing applications, it is becoming more and more important to have a system that infers by itself the most important features for the classification task and that can be updated in a fast and efficient way. Many works on neural networks demonstrated the ability of these algorithms to supply good classification performances even if the classification problem is so complex that cannot be parameterized by the user, as shown in computer vision, see e.g. applications in industrial objects classification [15], underwater [7] and outdoors images [5] interpretation. However, for almost all the supervised neural network typologies, the information is distributed on all the network, and so no new information can be learnt without destroying the old one. This means that every time a neural network has been trained on a given task and supplies good results, but there is the need to add other classes or, in general, more information to the training set, a new training of all the system is necessary; this operation can be time consuming and it is not assured that the performances on the old classes remain the same.
Classical methods like the decision trees, on the other side, are more easily updated by a restructuring procedure. In [17], [11], a method for incorporating new training instances after the first training of the system (ITI) is presented and applied [11] to image classification task. However, these methods need a detailed knowledge of the problem in order to select what are the most important features and what kind of tests should be done on them because, unlike neural networks, it is not possible to use a very large set of features.
A method that combines the advantages of both classical and neural models allowing an update of neural trees is presented in the following. A neural tree is a hybrid concept between the decision trees and the neural networks, characterized by neural networks instead of decision nodes in the inner nodes of the tree structure; many different models of neural trees have been developed in the past years (see e.g. [10], [13], [14]). The neural tree adopted here [8] is supervised and is characterized by a tree structure with perceptrons in the internal nodes. A detailed description of this neural tree is presented in Section 2. This kind of neural tree, like all the other neural trees composed by supervised neural networks, does not allow an update procedure; when it is necessary to add new information, a new training of the whole tree is needed.
However, the information is not distributed in all the structure, because different classes belong to different branches of the tree; it is therefore possible to modify a part of the neural structure without affecting the other parts. The algorithm proposed, named Information Update Algorithm for Neural Trees (IUANT) is described in 3 Algorithm description, 4 Update test and growing procedure, 5 Performance evaluation, while in Section 6 some experimental results are discussed.
Section snippets
The neural tree
In this section, a description of the adopted neural tree (widely described in [8]) is presented.
Algorithm description
The neural tree update is performed by testing the existing tree by patterns containing new information and adding, if necessary, some new branches generated by the training of these patterns in the right positions of the existing tree. The main idea of this paper is that, in general, the patterns containing the new information do not influence the whole tree.
Let T be the existing neural tree trained by a training set P={(p1,ω1),(p2,ω2)…(pn,ωn)} where pi is the ith pattern of the training set
Update test and growing procedure
When the classification is performed on real data, due to noise, there could be many patterns that present characteristics different from the ones of the classes to which they belong. A neural tree update using information carried by these noisy patterns may cause more mistakes in classification than the ones made by the original neural tree. For this reason, it is very important to understand when it is better not to update the neural tree. A first test, called update test, is made immediately
Performance evaluation
The performances evaluation method used by IUANT is the one adopted in the Enhanced Learning for Evolutive Neural Architectures (ELENA) European Research project [3]. One of the task of the project was to provide a set of databases and evaluation methods to be used for tests of classification algorithms. The apparent confusion matrix is defined as , where xik is the kth pattern in the test-set belonging to the class is the number of pattern belonging to the
Experimental results
In order to have a database independent evaluation of the IUANT performances, experiments have been carried on four real databases (SQUID, Satimage, Iris and Texture: see later for details), containing different kind of data, dimension and number of patterns. The performances of the tree obtained by IUANT, after the updating of an existing tree by a new training set P′, are compared with the performances of a new tree obtained directly from the training set P′ (i.e. destroying existing
Conclusions
In this paper, a new method named IUANT for information updating in supervised neural trees is presented. A system composed by the neural networks, that can be updated by the IUANT method, has the advantage to supply a self-evaluation of the most interesting features (like neural trees) and to allow an update without loosing of information (like ITI method applied to decision trees). The growing procedure proposed, together with the growing and update test, allows the growing only in presence
Acknowledgements
I would like to thank Dr. Walter Vanzella and Dr. Riccardo Giannitrapani for their valuable discussions and suggestions. I acknowledge Mr. Sadegh Abbasi, Dr. Farzin Mokhtarian, and Professor Josef Kittler of the Department of Electronic and Electrical Engineering at the University of Surrey for the image data-base SQUID and all the other people supplying the databases used in this paper (see references). I thank the reviewers who helped me in making this paper more readable and complete.
Stefania Gentili was born in Pisa (Italy) in 1969. She received her Laurea (Master) degree in Physics from the University of Pisa in 1995. Her thesis was about an automatic astronomical image analysis for the research of supernovae explosions in the ambit of an international research project named SWIRT. In 1996 she had a fellowship of Teramo Astronomical Observatory to work on images acquisition and analysis. From June 1997 she works in Udine University in the Industrial Computer Science
References (19)
- S. Gentili, Information update on neural tree networks, Proceedings of ICIP-2001 International Conference on Image...
- et al.
Structure-driven induction of decision tree classifiers through neural learning
Pattern Recognition
(1997) - et al.
Bias in information-based measures in decision tree induction
Machine Learning
(1994) The Irises of the Gaspe Peninsula (Original source)
Bull. Amer. Iris Soc.
(1935)Anonymous F.T.P.: /pub/machine-learning-database on...P.M. Murphy, D.W. Aha, Uci repository of machine learning databases, and Domain Theories,...- Australian Centre for Remote sensing (Original...Anonymous FTP: /pub/machine-learning-database on...P.M. Murphy, D.W. Aha, Uci repository of machine learning databases, and Domain Theories,...
- C. Aviles-Cruz, A. Guèrin-Duguè, J.L. Voz, D. Van Cappel, Deliverable R3-B1-P Task B1: Databases ELENA-NERVES-2...
TexturesA Photographic Album for Artists and Designers
(1966)- et al.
Interpreting image databases by region classification, Pattern Recognition
(1997) - W. Daelemans, J. Zavrel, K. van der Sloot, A. van den Bosch, TiMBL: Tilburg Memory Based Learner, version 4.1,...
Cited by (4)
An ensemble of RBF neural networks in decision tree structure with knowledge transferring to accelerate multi-classification
2019, Neural Computing and ApplicationsA neural-tree-based system for automatic location of earthquakes in Northeastern Italy
2006, Journal of SeismologyAutomatic picking of P and S phases using a neural tree
2006, Journal of SeismologySeismic monitoring in northeastern Italy: A ten-year experience
2005, Seismological Research Letters
Stefania Gentili was born in Pisa (Italy) in 1969. She received her Laurea (Master) degree in Physics from the University of Pisa in 1995. Her thesis was about an automatic astronomical image analysis for the research of supernovae explosions in the ambit of an international research project named SWIRT. In 1996 she had a fellowship of Teramo Astronomical Observatory to work on images acquisition and analysis. From June 1997 she works in Udine University in the Industrial Computer Science Laboratory with contracts, research fellowships or as Research Associate. Now, she has a fellowship from Italian Ministry of University and Scientific Research to have a Ph.D. in Computer Science. Recently she has been working at the EC project HOLOMAR, on the automatic classification of holographic images of plankton by neural networks and on two projects financed by CEOM (CEntro Oceanologico Mediterraneo), on the visual part of an autonomous underwater vehicle driving system. Her studies involved theoretical aspects and application of neural networks and invariant shape description. She was involved in the proposal of the EC project VENFLEX for the automatic recognition by neural networks of parts of furs and other flexible materials. She is referee of the journal “IEEE Transaction on Systems, Man and Cybernetics”. She is a member of the SAIt (Società Astronomica Italiana).