A novel approach to neuro-fuzzy classification
Introduction
Many attempts have been made in the last decades to design hybrid systems for pattern classification by combining the merits of individual techniques. An integration of neural networks (NNs) and fuzzy systems is one such hybrid technique and is known as neuro-fuzzy (NF) computing (Abe, 2001, Pal and Ghosh, 1996, Pal and Mitra, 1999).
Uncertainties can arise at any stage of a pattern classification system, resulting from incomplete or imprecise input information, ambiguity or vagueness in input data, ill-defined and/or overlapping boundaries among classes or regions, and indefiniteness in defining/extracting features and relations among them. It is therefore necessary for a classification system to have sufficient provision for representing uncertainties involved at every stage so that the final output (results) of the system is associated with the least possible uncertainty. The uncertainty handling issue becomes more prominent in case of land cover classification of remote sensing imagery (Richards and Jia, 2006, Tso and Mather, 2001, Varshney and Arora, 2004).
Since the fuzzy set theory (Zadeh, 1965) is a generalization of the classical set theory, it has greater flexibility to capture various aspects of incompleteness or imperfection about real life situations. The significance of fuzzy set theory in the realm of pattern classification is effectively justified in various areas such as representing input patterns as an array of membership values denoting the degree of possession of certain properties, representing linguistically defined input features, representing multiclass membership of ambiguous patterns, generating rules and inferences in linguistic form, extracting ill-defined image regions, and describing relations among them (Pal et al., 2000, Pedrycz, 1990).
Neural networks (NNs) are aimed at emulating the biological nervous system with the hope of achieving human-like performance artificially by capturing the key ingredients responsible for the remarkable capabilities of the human nervous system (Anthony and Bartlett, 1999, Haykin, 1997, Ripley, 1996, Rumelhart et al., 1986). Interaction among the neurons is very high in NNs making them suitable for collective decision making. The main characteristics of NNs, namely, adaptivity, fault tolerance, robustness and optimality play important roles particularly in the field of pattern classification.
Both NNs and fuzzy systems are adaptive in the estimation of the input–output function without any precise mathematical model. NNs handle numeric and quantitative information while fuzzy systems can handle symbolic and qualitative data. Apart from this, in a fuzzy classifier patterns are assigned with a degree of belonging to different classes. Thus the partitions in fuzzy classifiers are soft and gradual rather than hard and crisp. Therefore, an integration of neural and fuzzy systems should have the merits of both and it should enable one to build more intelligent decision making systems. Fuzzy set theory based hybrid classification systems are found to be more suitable and appropriate to handle these situations reasonably (Kuncheva, 2000, Pedrycz, 1990).
In the NF paradigm, much research effort has been made (Abe, 2001, Baraldi et al., 2001, Boskovitz and Guterman, 2002, Gamba and Dellacqua, 2003, Ghosh et al., 1993, Han et al., 2002, Keller and Hunt, 1985, Kwon et al., 1994, Pal and Ghosh, 1996, Pal and Mitra, 1999, Qiu and Jensen, 2004). NF hybridization is done broadly in two ways: NNs that are capable of handling fuzzy information (named as fuzzy-neural networks (FNN)), and fuzzy systems augmented by NNs to enhance some of their characteristics such as flexibility, speed and adaptability (named as neural-fuzzy systems (NFS)) (Pal and Ghosh, 1996, Pal and Mitra, 1999). Other than these two, fuzzy sets/logic can also be incorporated in NNs in various ways. All these methodologies can be broadly categorized into five NF integration procedures, and the details on these methodologies can be found in Pal and Ghosh (1996).
The main aim of the present work is to explore various possible degrees of belonging of all features independently to different classes; normally not used in conventional NF classification systems. The proposed hybrid classification model assigns memberships for each feature of a pattern to different classes forming the membership matrix. The number of columns and rows of the matrix are equal to the number of classes and number of features (spectral bands for remote sensing images), respectively. Therefore, the input vector will have a dimension equal to the product of the number of classes and the number of features. In other words, the number of input nodes of the NN is equal to the number of elements of the membership matrix. This membership matrix is converted into a vector by cascading all rows (columns) and becomes the input to the NN. Number of output nodes of the NN is equal to the number of classes. Defuzzification operation is then performed on the NN output. A hard classification of the input pattern can be obtained using a MAX (maximum) operation on the output of NN as in the case of a conventional fuzzy classification system.
The organization of rest of the article is as follows. A detailed description of the proposed NF classification model has been made in Section 2. Section 3 describes the experimental results with comparative analysis. Various performance measures used in the present investigation are also discussed in this section. Finally, concluding remarks are provided in Section 4.
Section snippets
Proposed neuro-fuzzy classification model
A new model for the neuro-fuzzy (NF) classification system is proposed in the present article. The proposed NF classification system extracts feature-wise information of input pattern to different classes. Since all features are not equally important in discriminating all classes, the feature-wise belonging is expected to help in the classification process. The block diagram of the proposed NF model is shown in Fig. 1.
The proposed model works in three steps. In the first step, the system takes
Experimental results and analysis
For establishing the usefulness of the proposed model we considered four conventional fully labeled data sets (including a satellite image data) and two partially labeled multi-spectral remote sensing images. A brief description of the conventional (labeled) data sets is given in Table 1, and the remote sensing images used are shown in Figs. 3(a) and 5(a).
Selection of the training and test samples for all classes in case of conventional (fully labeled) data sets have been made after dividing
Conclusion
We have proposed a novel neuro-fuzzy model for classification and demonstrated successfully its effectiveness for classification of fully and partially labeled patterns. The method exploits and incorporates the basic advantages of neural networks such as massive parallelism, robustness, adaptivity and optimality in one hand; and impreciseness and uncertainty handling capability of fuzzy sets on the other hand. Besides these generic advantages, the proposed model develops a membership matrix
Acknowledgements
The authors would like to thank the reviewers for their valuable and constructive suggestions. Thanks are also due to the Department of Science and Technology, Government of India, under which a project titled “Advanced Techniques for Remote Sensing Image Processing” is being carried out at the Machine Intelligence Unit, Indian Statistical Institute, Kolkata.
References (38)
A review of assessing the accuracy of classifications of remotely sensed data
Remote Sensing of Environment
(1991)- et al.
A novel fuzzy classifier based on product aggregation operator
Pattern Recognition
(2008) - et al.
Segmentation of multispectral remote sensing images using active support vector machines
Pattern Recognition Letters
(2004) Fuzzy sets in pattern recognition: Methodology and methods
Pattern Recognition
(1990)Fuzzy sets
Information Control
(1965)Pattern classification: Neuro-fuzzy methods and their comparison
(2001)- et al.
Segmentation of remotely sensed images using wavelet features and their evaluation in soft computing framework
IEEE Transactions on Geoscience and Remote Sensing
(2003) - et al.
Neural network learning: Theoretical foundations
(1999) - Asuncion, A., & Newman, D. (2007). UCI machine learning repository. University of California, Irvine, School of...
- et al.
Learning in linear neural networks: A survey
IEEE Transactions on Neural Networks
(1995)