Elsevier

Neural Networks

Volume 22, Issue 1, January 2009, Pages 100-109
Neural Networks

A novel approach to neuro-fuzzy classification

https://doi.org/10.1016/j.neunet.2008.09.011Get rights and content

Abstract

A new model for neuro-fuzzy (NF) classification systems is proposed. The motivation is to utilize the feature-wise degree of belonging of patterns to all classes that are obtained through a fuzzification process. A fuzzification process generates a membership matrix having total number of elements equal to the product of the number of features and classes present in the data set. These matrix elements are the input to neural networks. The effectiveness of the proposed model is established with four benchmark data sets (completely labeled) and two remote sensing images (partially labeled). Different performance measures such as misclassification, classification accuracy and kappa index of agreement for completely labeled data sets, and β index of homogeneity and Davies–Bouldin (DB) index of compactness for remotely sensed images are used for quantitative analysis of results. All these measures supported the superiority of the proposed NF classification model. The proposed model learns well even with a lower percentage of training data that makes the system fast.

Introduction

Many attempts have been made in the last decades to design hybrid systems for pattern classification by combining the merits of individual techniques. An integration of neural networks (NNs) and fuzzy systems is one such hybrid technique and is known as neuro-fuzzy (NF) computing (Abe, 2001, Pal and Ghosh, 1996, Pal and Mitra, 1999).

Uncertainties can arise at any stage of a pattern classification system, resulting from incomplete or imprecise input information, ambiguity or vagueness in input data, ill-defined and/or overlapping boundaries among classes or regions, and indefiniteness in defining/extracting features and relations among them. It is therefore necessary for a classification system to have sufficient provision for representing uncertainties involved at every stage so that the final output (results) of the system is associated with the least possible uncertainty. The uncertainty handling issue becomes more prominent in case of land cover classification of remote sensing imagery (Richards and Jia, 2006, Tso and Mather, 2001, Varshney and Arora, 2004).

Since the fuzzy set theory (Zadeh, 1965) is a generalization of the classical set theory, it has greater flexibility to capture various aspects of incompleteness or imperfection about real life situations. The significance of fuzzy set theory in the realm of pattern classification is effectively justified in various areas such as representing input patterns as an array of membership values denoting the degree of possession of certain properties, representing linguistically defined input features, representing multiclass membership of ambiguous patterns, generating rules and inferences in linguistic form, extracting ill-defined image regions, and describing relations among them (Pal et al., 2000, Pedrycz, 1990).

Neural networks (NNs) are aimed at emulating the biological nervous system with the hope of achieving human-like performance artificially by capturing the key ingredients responsible for the remarkable capabilities of the human nervous system (Anthony and Bartlett, 1999, Haykin, 1997, Ripley, 1996, Rumelhart et al., 1986). Interaction among the neurons is very high in NNs making them suitable for collective decision making. The main characteristics of NNs, namely, adaptivity, fault tolerance, robustness and optimality play important roles particularly in the field of pattern classification.

Both NNs and fuzzy systems are adaptive in the estimation of the input–output function without any precise mathematical model. NNs handle numeric and quantitative information while fuzzy systems can handle symbolic and qualitative data. Apart from this, in a fuzzy classifier patterns are assigned with a degree of belonging to different classes. Thus the partitions in fuzzy classifiers are soft and gradual rather than hard and crisp. Therefore, an integration of neural and fuzzy systems should have the merits of both and it should enable one to build more intelligent decision making systems. Fuzzy set theory based hybrid classification systems are found to be more suitable and appropriate to handle these situations reasonably (Kuncheva, 2000, Pedrycz, 1990).

In the NF paradigm, much research effort has been made (Abe, 2001, Baraldi et al., 2001, Boskovitz and Guterman, 2002, Gamba and Dellacqua, 2003, Ghosh et al., 1993, Han et al., 2002, Keller and Hunt, 1985, Kwon et al., 1994, Pal and Ghosh, 1996, Pal and Mitra, 1999, Qiu and Jensen, 2004). NF hybridization is done broadly in two ways: NNs that are capable of handling fuzzy information (named as fuzzy-neural networks (FNN)), and fuzzy systems augmented by NNs to enhance some of their characteristics such as flexibility, speed and adaptability (named as neural-fuzzy systems (NFS)) (Pal and Ghosh, 1996, Pal and Mitra, 1999). Other than these two, fuzzy sets/logic can also be incorporated in NNs in various ways. All these methodologies can be broadly categorized into five NF integration procedures, and the details on these methodologies can be found in Pal and Ghosh (1996).

The main aim of the present work is to explore various possible degrees of belonging of all features independently to different classes; normally not used in conventional NF classification systems. The proposed hybrid classification model assigns memberships for each feature of a pattern to different classes forming the membership matrix. The number of columns and rows of the matrix are equal to the number of classes and number of features (spectral bands for remote sensing images), respectively. Therefore, the input vector will have a dimension equal to the product of the number of classes and the number of features. In other words, the number of input nodes of the NN is equal to the number of elements of the membership matrix. This membership matrix is converted into a vector by cascading all rows (columns) and becomes the input to the NN. Number of output nodes of the NN is equal to the number of classes. Defuzzification operation is then performed on the NN output. A hard classification of the input pattern can be obtained using a MAX (maximum) operation on the output of NN as in the case of a conventional fuzzy classification system.

The organization of rest of the article is as follows. A detailed description of the proposed NF classification model has been made in Section 2. Section 3 describes the experimental results with comparative analysis. Various performance measures used in the present investigation are also discussed in this section. Finally, concluding remarks are provided in Section 4.

Section snippets

Proposed neuro-fuzzy classification model

A new model for the neuro-fuzzy (NF) classification system is proposed in the present article. The proposed NF classification system extracts feature-wise information of input pattern to different classes. Since all features are not equally important in discriminating all classes, the feature-wise belonging is expected to help in the classification process. The block diagram of the proposed NF model is shown in Fig. 1.

The proposed model works in three steps. In the first step, the system takes

Experimental results and analysis

For establishing the usefulness of the proposed model we considered four conventional fully labeled data sets (including a satellite image data) and two partially labeled multi-spectral remote sensing images. A brief description of the conventional (labeled) data sets is given in Table 1, and the remote sensing images used are shown in Figs. 3(a) and 5(a).

Selection of the training and test samples for all classes in case of conventional (fully labeled) data sets have been made after dividing

Conclusion

We have proposed a novel neuro-fuzzy model for classification and demonstrated successfully its effectiveness for classification of fully and partially labeled patterns. The method exploits and incorporates the basic advantages of neural networks such as massive parallelism, robustness, adaptivity and optimality in one hand; and impreciseness and uncertainty handling capability of fuzzy sets on the other hand. Besides these generic advantages, the proposed model develops a membership matrix

Acknowledgements

The authors would like to thank the reviewers for their valuable and constructive suggestions. Thanks are also due to the Department of Science and Technology, Government of India, under which a project titled “Advanced Techniques for Remote Sensing Image Processing” is being carried out at the Machine Intelligence Unit, Indian Statistical Institute, Kolkata.

References (38)

  • A. Baraldi et al.

    Comparison of the multilayer perceptron with neuro-fuzzy techniques in the estimation of cover class mixture in remotely sensed data

    IEEE Transaction on Geoscience and Remote Sensing

    (2001)
  • V. Boskovitz et al.

    An adaptive neuro-fuzzy system for automatic image segmentation and edge detection

    IEEE Transaction on Fuzzy Systems

    (2002)
  • D.H. Card

    Using known map category marginal frequencies to improve estimates of thematic map accuracy

    Photogrammetric Engineering and Remote Sensing

    (1982)
  • D.L. Davies et al.

    A cluster separation measure

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1979)
  • ELENA Database (1995). Deliverable R3-B1-P Task B1: Databases (ESPRIT Basic Research Project Number 6891), MLG, UCL,...
  • P. Gamba et al.

    Increased accuracy multiband urban classification using a neuro-fuzzy classifier

    International Journal of Remote Sensing

    (2003)
  • A. Ghosh et al.

    Self-organization for object extraction using a multilayer neural network and fuzziness measures

    IEEE Transaction on Fuzzy Systems

    (1993)
  • Han, J., Lee, S., Chi, K., & Ryu, K. (2002) Comparison of neuro-fuzzy, neural network, and maximum likelihood...
  • S. Haykin

    Neural networks: A comprehensive foundation

    (1997)
  • Cited by (0)

    View full text