Elsevier

Neurocomputing

Volume 51, April 2003, Pages 61-74
Neurocomputing

A new method for information update in supervised neural structures

https://doi.org/10.1016/S0925-2312(02)00602-1Get rights and content

Abstract

Supervised neural networks can evaluate the most important features and the relations between them from a redundant set, while they allow the control on how to group the patterns. However, they do not allow an updating of the stored information without destroying the existing one. The method proposed in this paper allows a fast update of neural trees composed by supervised neural networks in their nodes; it is robust to noise and supplies good performances, in comparison with a new training, both in classification and in time required. Some tests are presented on four public domain real databases.

Introduction

Due to the increasing number of available data that must be analyzed and classified in modern computing applications, it is becoming more and more important to have a system that infers by itself the most important features for the classification task and that can be updated in a fast and efficient way. Many works on neural networks demonstrated the ability of these algorithms to supply good classification performances even if the classification problem is so complex that cannot be parameterized by the user, as shown in computer vision, see e.g. applications in industrial objects classification [15], underwater [7] and outdoors images [5] interpretation. However, for almost all the supervised neural network typologies, the information is distributed on all the network, and so no new information can be learnt without destroying the old one. This means that every time a neural network has been trained on a given task and supplies good results, but there is the need to add other classes or, in general, more information to the training set, a new training of all the system is necessary; this operation can be time consuming and it is not assured that the performances on the old classes remain the same.

Classical methods like the decision trees, on the other side, are more easily updated by a restructuring procedure. In [17], [11], a method for incorporating new training instances after the first training of the system (ITI) is presented and applied [11] to image classification task. However, these methods need a detailed knowledge of the problem in order to select what are the most important features and what kind of tests should be done on them because, unlike neural networks, it is not possible to use a very large set of features.

A method that combines the advantages of both classical and neural models allowing an update of neural trees is presented in the following. A neural tree is a hybrid concept between the decision trees and the neural networks, characterized by neural networks instead of decision nodes in the inner nodes of the tree structure; many different models of neural trees have been developed in the past years (see e.g. [10], [13], [14]). The neural tree adopted here [8] is supervised and is characterized by a tree structure with perceptrons in the internal nodes. A detailed description of this neural tree is presented in Section 2. This kind of neural tree, like all the other neural trees composed by supervised neural networks, does not allow an update procedure; when it is necessary to add new information, a new training of the whole tree is needed.

However, the information is not distributed in all the structure, because different classes belong to different branches of the tree; it is therefore possible to modify a part of the neural structure without affecting the other parts. The algorithm proposed, named Information Update Algorithm for Neural Trees (IUANT) is described in 3 Algorithm description, 4 Update test and growing procedure, 5 Performance evaluation, while in Section 6 some experimental results are discussed.

Section snippets

The neural tree

In this section, a description of the adopted neural tree (widely described in [8]) is presented.

Algorithm description

The neural tree update is performed by testing the existing tree by patterns containing new information and adding, if necessary, some new branches generated by the training of these patterns in the right positions of the existing tree. The main idea of this paper is that, in general, the patterns containing the new information do not influence the whole tree.

Let T be the existing neural tree trained by a training set P={(p1,ω1),(p2,ω2)…(pn,ωn)} where pi is the ith pattern of the training set

Update test and growing procedure

When the classification is performed on real data, due to noise, there could be many patterns that present characteristics different from the ones of the classes to which they belong. A neural tree update using information carried by these noisy patterns may cause more mistakes in classification than the ones made by the original neural tree. For this reason, it is very important to understand when it is better not to update the neural tree. A first test, called update test, is made immediately

Performance evaluation

The performances evaluation method used by IUANT is the one adopted in the Enhanced Learning for Evolutive Neural Architectures (ELENA) European Research project [3]. One of the task of the project was to provide a set of databases and evaluation methods to be used for tests of classification algorithms. The apparent confusion matrix Ĉij is defined as Ĉij=(100/Ni)∑k=1Nifj(xik), where xik is the kth pattern in the test-set belonging to the class i,Ni is the number of pattern belonging to the

Experimental results

In order to have a database independent evaluation of the IUANT performances, experiments have been carried on four real databases (SQUID, Satimage, Iris and Texture: see later for details), containing different kind of data, dimension and number of patterns. The performances of the tree obtained by IUANT, after the updating of an existing tree by a new training set P′, are compared with the performances of a new tree obtained directly from the training set P′ (i.e. destroying existing

Conclusions

In this paper, a new method named IUANT for information updating in supervised neural trees is presented. A system composed by the neural networks, that can be updated by the IUANT method, has the advantage to supply a self-evaluation of the most interesting features (like neural trees) and to allow an update without loosing of information (like ITI method applied to decision trees). The growing procedure proposed, together with the growing and update test, allows the growing only in presence

Acknowledgements

I would like to thank Dr. Walter Vanzella and Dr. Riccardo Giannitrapani for their valuable discussions and suggestions. I acknowledge Mr. Sadegh Abbasi, Dr. Farzin Mokhtarian, and Professor Josef Kittler of the Department of Electronic and Electrical Engineering at the University of Surrey for the image data-base SQUID and all the other people supplying the databases used in this paper (see references). I thank the reviewers who helped me in making this paper more readable and complete.

Stefania Gentili was born in Pisa (Italy) in 1969. She received her Laurea (Master) degree in Physics from the University of Pisa in 1995. Her thesis was about an automatic astronomical image analysis for the research of supernovae explosions in the ambit of an international research project named SWIRT. In 1996 she had a fellowship of Teramo Astronomical Observatory to work on images acquisition and analysis. From June 1997 she works in Udine University in the Industrial Computer Science

References (19)

  • S. Gentili, Information update on neural tree networks, Proceedings of ICIP-2001 International Conference on Image...
  • I. Sethi et al.

    Structure-driven induction of decision tree classifiers through neural learning

    Pattern Recognition

    (1997)
  • A.P. White et al.

    Bias in information-based measures in decision tree induction

    Machine Learning

    (1994)
  • E. Andersen

    The Irises of the Gaspe Peninsula (Original source)

    Bull. Amer. Iris Soc.

    (1935)
    Anonymous F.T.P.: /pub/machine-learning-database on...P.M. Murphy, D.W. Aha, Uci repository of machine learning databases, and Domain Theories,...
  • Australian Centre for Remote sensing (Original...Anonymous FTP: /pub/machine-learning-database on...P.M. Murphy, D.W. Aha, Uci repository of machine learning databases, and Domain Theories,...
  • C. Aviles-Cruz, A. Guèrin-Duguè, J.L. Voz, D. Van Cappel, Deliverable R3-B1-P Task B1: Databases ELENA-NERVES-2...
  • P. Brodatz

    TexturesA Photographic Album for Artists and Designers

    (1966)
  • N.W. Campbell et al.

    Interpreting image databases by region classification, Pattern Recognition

    (1997)
  • W. Daelemans, J. Zavrel, K. van der Sloot, A. van den Bosch, TiMBL: Tilburg Memory Based Learner, version 4.1,...
There are more references available in the full text version of this article.

Stefania Gentili was born in Pisa (Italy) in 1969. She received her Laurea (Master) degree in Physics from the University of Pisa in 1995. Her thesis was about an automatic astronomical image analysis for the research of supernovae explosions in the ambit of an international research project named SWIRT. In 1996 she had a fellowship of Teramo Astronomical Observatory to work on images acquisition and analysis. From June 1997 she works in Udine University in the Industrial Computer Science Laboratory with contracts, research fellowships or as Research Associate. Now, she has a fellowship from Italian Ministry of University and Scientific Research to have a Ph.D. in Computer Science. Recently she has been working at the EC project HOLOMAR, on the automatic classification of holographic images of plankton by neural networks and on two projects financed by CEOM (CEntro Oceanologico Mediterraneo), on the visual part of an autonomous underwater vehicle driving system. Her studies involved theoretical aspects and application of neural networks and invariant shape description. She was involved in the proposal of the EC project VENFLEX for the automatic recognition by neural networks of parts of furs and other flexible materials. She is referee of the journal “IEEE Transaction on Systems, Man and Cybernetics”. She is a member of the SAIt (Società Astronomica Italiana).

View full text