Elsevier

Pattern Recognition

Volume 41, Issue 3, March 2008, Pages 961-971
Pattern Recognition

A novel fuzzy classifier based on product aggregation operator

https://doi.org/10.1016/j.patcog.2007.08.002Get rights and content

Abstract

The present article proposes a fuzzy set-based classifier with a better learning and generalization capability. The proposed classifier exploits the feature-wise degree of belonging of a pattern to all classes, generalization in the fuzzification process and the combined class-wise contribution of features effectively. The classifier uses a π-type membership function and product aggregation reasoning rule (operator). Its effectiveness is verified with two conventional (completely labeled) data sets and two remote sensing images (partially labeled data sets). The proposed classifier is compared with similar fuzzy methods. Different performance measures are used for quantitative evaluation of the proposed classifier.

Introduction

Classification of patterns [1], [2] is an important area in a variety of fields including artificial intelligence [3] computer vision [4] and image analysis [5]. In such problems, if a priori probabilities and the conditional probability density of all classes are known, then Bayes decision theory produces optimal results [1], [2], i.e., it provides minimum expected error. However, in many applications, such knowledge is not available. For these cases algorithms like maximum likelihood (ML) [1], k-nearest neighbor [1], [2] and the soft computing tools like neural networks (NNs) [6], [7], fuzzy sets [8], [9], [10], genetic algorithms [11] are used.

A conventional hard or non-fuzzy classifier assumes that the pattern x belongs to a particular class only according to the given criteria. The hard classifiers are thus easy to implement and can be used to classify the classes that are well separable, well defined and have distinct boundaries. However, these algorithms may not be useful to classify ill-defined with overlapping classes. For such problems fuzzy classifiers [8], [12] are more useful as it allows imprecise class definition and recognize patterns belonging to more than one class [13], [14] with varying degree of membership values. Thus the partitions in fuzzy classes are soft and gradual rather than hard and crisp. With the coming of fuzzy sets [8], many research works have been carried out for applications to pattern classification and decision making problems. The most important work done in this area includes fuzzy k-nearest neighbor algorithm by Keller et al. [15], fuzzy rule-based algorithms by Ishibuchi et al. [16], Abe and Lan [17] and fuzzy ML classifier by Wang [18]. In this regard, Pedrycz [19] provided a survey on fuzzy classification methods. Fuzzy techniques are applied successfully to various areas including land cover classification of remote sensing images [20], [21]. A summary of different fuzzy classifiers and their applications are described by Kuncheva [12].

Fuzzy rule-based classification systems have become an important research area in the recent past. Many approaches have been proposed for generating and learning fuzzy if-then rules from numerical data for classification problems [9], [10], [16], [17], [22], [23], [24]. A comparative analysis has been made by Ishibuchi and Yamamoto [22] on heuristic criteria that are used for extracting a pre-specified number of fuzzy classification rules from numerical data. In a similar study, Abe and Lan [17] described a method of extracting fuzzy rules directly from numerical input–output data for pattern classification. These rules are extracted from numerical data by recursively resolving overlaps between two classes. Then, optimal number of input variables for the rules are determined using the number of extracted rules as a criterion. In another approach Ishibuchi and Nakashima [25] proposed to use the effects of rule weights in fuzzy rule-based classification systems. Further, Bardossy and Samaniego [23] proposed a fuzzy rule-based classification of remotely sensed images. Here they have used a simulated annealing-based optimization technique to derive the fuzzy rules from training data. A support vector learning for fuzzy rule-based classification systems is proposed by Chen and Wang [24], where they discussed the connection between fuzzy classifiers and kernel machines that establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. Apart from these, in some approaches aggregation operators are used on the fuzzified value to get an aggregated decision on the available information [9], [10], [26]. Peneva and Popchev [27] described an application of different fuzzy logic operators in decision making and discussed how to enhance the ability to solve the problem of ranking or choice. However, the selection of an aggregation operator is an important issue in any decision making process. In this regard, Beliakov and Warren [28] suggested a few ways of selecting aggregation operators in fuzzy decision support systems. A large variety of aggregation operators has been proposed by Bloch [29]. She made a classification of these operators used in different data fusion theories with respect to their behavior and the classification provides a guide for choosing an operator for a given problem. In another study, a discussion is provided by Dubois et al. [26] to suggest directions for using the results of mathematical investigations in the structure of aggregation operations. The problem ofinformation generalization in multi-criteria decision making, where the information is unified by fuzzy relations, is realized with the help of aggregation operators [27]. However, the proposed classifier is different from the above mentioned approaches.

Two important aspects, namely learning and generalization capabilities, play an important role in any pattern classification problem. Intuitively, these can be achieved through feature-wise information extraction, generalization in the fuzzification process and combined contribution of these information to all classes of a pattern because there is a high possibility that various valuable information for different classes may reside in features of a pattern and they supplement each other. The problem becomes more complex if the classes are overlapping and ill-defined. Keeping in view of these aspects, we have designed a classifier and highlighted the method of feature-wise extraction of information and combining/aggregating the features’ information to get an improved classification. The actual parameters that are participating in the design and classification process are the membership value of the features to different classes, which in turn shows how much a feature is compatible to a class. The objective of the product aggregation is to assign a pattern to a class where all the features are useful to represent that class properly, rather than the class where only some features are representing it. These characteristics are very useful in remotely sensed image analysis. In some other real life problems this may not be true. In such cases we may use some feature selection methods to choose a set of features, all of which contribute in designing the classifier. Alternatively, we can use different aggregation rules, which may be suitable for the problem at hand. The product aggregation rule is applicable for problems where the fuzzy sets (with respect to each feature) represent properties, all of which contribute to a large extent to the desired class [12], [26], [29], [30], [31]. This is applicable in case of fuzzy sets only and not in case of crisp sets. Due to overlapping nature of the classes that we normally consider in real life applications, this type of decision is very much suitable for fusion of the feature-wise information.

In addition to this, we have also taken care of the generalization capability of classifiers to further enhance the classification performance. In this regard, we have proposed a fuzzy product aggregation reasoning rule-based classifier and applied on two conventional (completely labeled) data sets and two remote sensing images (partially labeled) to justify its potentiality. Various performance measurement parameters such as number of overall misclassification, percentage of overall accuracy, producer's accuracy [32], user's accuracy [32] are considered for completely labeled data sets. For remote sensing images, β index [33] of homogeneity and Xie–Beni (XB) [34] index of compactness are evaluated to validate the superiority of the proposed classifier over others. In addition to these performance measures, Kappa coefficient (KC) [35] is also estimated for completely labeled data sets to corroborate the advantage of the proposed fuzzy classifier. Experimental results showed promising and improved classification performance on the above mentioned data sets.

The objective of the present article is to demonstrate the usefulness of the fuzzy product aggregation reasoning rule in classification of remote sensing images. Therefore, we have used the spectral (band) values as feature values. Each of these feature values is used to generate class-wise membership values which are used as final features. Also we have used two conventional (completely labeled) data sets (i.e., WAVEFORM and BUPA [36]) to demonstrate the effectiveness of the proposed method.

The organization of the article is as follows. Section 2 describes the proposed classifier and discusses the advantages. Different performance measurement parameters are discussed in Section 3. A brief description on the data sets used is given in Section 4. In Section 5, a complete discussion on the implementation and results are given. Finally, the concluding remarks are given in Section 6.

Section snippets

Proposed fuzzy classifier

The proposed fuzzy classifier has three steps of operation as illustrated in Fig. 1. The first step fuzzifies the input feature vector using a π-type membership function (MF) [8] that explores the information of different features for each pattern and collects the hidden or interrelated information to provide a better classification accuracy. The advantage of using π-type MF is that it has a parameter, called fuzzifier (N), which can be tuned easily according to the requirement of the problem.

Performance measurement indices

To examine the practical applicability of the proposed classifier, various performance measures are used for conventional data sets. These are number of misclassification (MC), percentage of overall accuracy (PA), producer's and user's accuracy and Kappa coefficient (KC). The MC value in the classification process is the number of overall samples/patterns that are wrongly classified. The PA value is the percentage of samples that are correctly classified. The MC and PA parameters are calculated

Description of the data set used

A short description of two conventional data sets, namely, WAVEFORM [36] and BUPA (liver disorder) [36], are used in the present study and are provided in Table 1. We have used data sets bearing different number of features and classes. Also the data sets are selected from the group of both small and large number of labeled patterns. In addition to these, two multispectral remote sensing images (size 512×512) obtained from two different satellites are used for the present simulation study: one

Strategy of selecting the training data set

The conventional data sets are divided into two parts. First part is taken for estimation of the parameters of the classifier (training data). The second part is taken for testing the performance (test data). We have taken three different sizes of data as training: these are 10%, 20% and 50% and the rest 90%, 80% and 50% considered as test data. Selection of the training data is random in nature and an equal percent of data is collected from each class.

For remote sensing images training samples

Conclusion

In the present article we have proposed a fuzzy classifier (based on fuzzy sets) that explored three important aspects. These are (i) extracted feature-wise information for different classes, (ii) generalization capability and (iii) combined contribution of individual features to a particular class. It is observed that individually the PROD aggregation reasoning rule (RR) has a better classification capability compared to other RRs. This is because of the fact that the fuzzy product aggregation

Acknowledgments

The authors would like to thank the reviewers for their valuable and constructive suggestions. Thanks are also due to the Department of Science and Technology, Government of India, under which a project titled “Advanced Techniques for Remote Sensing Image Processing” is being carried out at the Machine Intelligence Unit, Indian Statistical Institute, Kolkata.

About the AuthorASHISH GHOSH is a Professor of the Machine Intelligence Unit at the Indian Statistical Institute, Kolkata. He received the B.E. degree in Electronics and Telecommunication from the Jadavpur University, Kolkata, in 1987, and the M.Tech. and Ph.D. degrees in Computer Science from the Indian Statistical Institute, Kolkata, in 1989 and 1993, respectively. He received the prestigious and most coveted Young Scientists award in Engineering Sciences from the Indian National Science

References (44)

  • A.B. Watson

    Digital Images and Human Vision

    (1993)
  • S. Haykin

    Neural Networks: A Comprehensive Foundation

    (1998)
  • B.D. Ripley

    Pattern Recognition and Neural Networks

    (1996)
  • H.J. Zimmermann

    Fuzzy Set Theory and its Applications

    (2001)
  • G.J. Klir et al.

    Fuzzy Sets and Fuzzy Logic: Theory and Applications

    (1997)
  • D.E. Goldberg

    Genetic Algorithms in Search, Optimization, and Machine Learning

    (1989)
  • L.I. Kuncheva

    Fuzzy Classifier Design

    (2000)
  • F. Maselli et al.

    Fuzzy classification of spatially degraded thematic mapper data for the estimation of sub-pixel components

    Int. J. Remote Sensing

    (1996)
  • D.P. Mandal et al.

    Formulation of a multivalued recognition system

    IEEE Trans. Syst. Man Cybern.

    (1992)
  • J.M. Keller et al.

    A fuzzy k-nearest neighbor algorithm

    IEEE Trans. Syst. Man Cybern.

    (1985)
  • S. Abe et al.

    A method for fuzzy rules extraction directly from numerical data and its application to pattern classification

    IEEE Trans. Fuzzy Syst.

    (1995)
  • F. Wang

    Fuzzy supervised classification of remote sensing images

    IEEE Trans. Geosci. Remote Sensing

    (1990)
  • Cited by (35)

    • Fuzzy classifiers with information granules in feature space and logic-based computing

      2018, Pattern Recognition
      Citation Excerpt :

      A concept of fuzzy classification and ensuing fuzzy classifiers go back to the seminal paper by Zadeh and Kalaba [1]. Since then there has been a growing interest in fuzzy classification, cf. [2–4,45]. In particular, fuzzy classifiers coming in the form of rule-based classification rules have attracted a particular attention both in terms of their functionality as well as ensuing analysis and design, see [3,5–10].

    • Using an unsupervised approach of Probabilistic Neural Network (PNN) for land use classification from multitemporal satellite images

      2015, Applied Soft Computing
      Citation Excerpt :

      The consistency of those models is evaluated based on the actual data. The most used supervised classification methods are: maximum likelihood classification (MLC) [12], parallelepiped method (PP) [13] and fuzzy sets [14-18], neural networks (NNs) [19,20], support vector machines (SVM) [21,22] and computational intelligence [23]. In other hand, the basic task of unsupervised learning methods is to develop classification labels automatically.

    • Explicit rough-fuzzy pattern classification model

      2014, Pattern Recognition Letters
    • Fuzzy clustering algorithms incorporating local information for change detection in remotely sensed images

      2012, Applied Soft Computing Journal
      Citation Excerpt :

      A few context-sensitive techniques using neural networks are also suggested recently [4,5,23,24]. Relevance of fuzzy set theoretic methods in pattern recognition and image analysis problems has adequately been addressed in the literature [25–35]. Fuzzy clustering incorporating local information for change detection in remotely sensed images has not been reported in the literature.

    View all citing articles on Scopus

    About the AuthorASHISH GHOSH is a Professor of the Machine Intelligence Unit at the Indian Statistical Institute, Kolkata. He received the B.E. degree in Electronics and Telecommunication from the Jadavpur University, Kolkata, in 1987, and the M.Tech. and Ph.D. degrees in Computer Science from the Indian Statistical Institute, Kolkata, in 1989 and 1993, respectively. He received the prestigious and most coveted Young Scientists award in Engineering Sciences from the Indian National Science Academy in 1995, and in Computer Science from the Indian Science Congress Association in 1992. He has been selected as an Associate of the Indian Academy of Sciences, Bangalore, in 1997. He visited the Osaka Prefecture University, Japan, with a Post-doctoral fellowship during October 1995 to March 1997, and Hannan University, Japan, as a Visiting Faculty during September to October, 1997 and September to October, 2004. He has also visited Hannan University, Japan, as Visiting Professor with a fellowship from Japan Society for Promotion of Sciences (JSPS) during February to April, 2005. During May 1999, he was at the Institute of Automation, Chinese Academy of Sciences, Beijing, with CIMPA (France) fellowship. He was at the German National Research Center for Information Technology, Germany, with a German Government (DFG) Fellowship during January to April, 2000. During October to December 2003 he was a Visiting Professor at the University of California, Los Angeles; and during December 2006 to January 2007 he was at the Computer Science Department of Yonsei University, Korea. His visits to University of Trento and University of Palermo (Italy) during May to June 2004, March to April 2006, and May to June 2007 were in connection with collaborative international projects. He also visited various Universities/Academic Institutes and delivered lectures in different countries including South Korea, Poland and The Netherland.

    His research interests include Pattern Recognition and Machine Learning, Data Mining, Image Analysis, Remotely Sensed Image Analysis, Soft Computing, Fuzzy Sets and Uncertainty Analysis, Neural Networks, Evolutionary Computation and Bioinformatics. He has already published about 100 research papers in internationally reputed journals and refereed conferences, has edited six books and is acting as a member of the editorial board of various international journals.

    He is a member of the founding team that established a National Center for Soft Computing Research at the Indian Statistical Institute, Kolkata, in 2004, with funding from the Department of Science and Technology (DST), Government of India.

    About the AuthorSAROJ K. MEHER received the Ph.D. degree in electronics engineering from National Institute of Technology, Rourkela, India, in 2003. He is a Post-Doctoral Fellow in the Machine Intelligence Unit, Indian Statistical Institute, Kolkata, India. His research interests include pattern recognition, soft computing methods, fuzzy sets and digital signal processing. He has published many research articles in internationally reputed journals and refereed conferences.

    About the AuthorB. UMA SHANKAR received the M.Sc. degree in statistics from Bhagalpur University, India, in 1985, and the Ph.D. degree in Science from Jadavpur University, India, in 2006. He is an Associate Scientist in the Machine Intelligence Unit, Indian Statistical Institute, Kolkata, India. He was at Stanford University, USA, from September 1992 to March 1993; at CIMPA, Nice, France, from June to July 1990, and June to July 1993, and at the University of Trento, Italy from January to March 2007. His research interests include pattern recognition and machine learning, analysis of remotely sensed images, fuzzy sets, rough sets and neural networks. He has published many research papers in international journals and refereed conferences and is acting as a reviewer of international journals.

    View full text