Abstract
Pairwise classification is a well-known class binarization technique that converts a multiclass problem into a number of two-class problems, one problem for each pair of classes. However, in the pairwise technique, nuisance votes of many irrelevant classifiers may result in a wrong class prediction. To overcome this problem, a simple, but efficient method is proposed and evaluated in this paper. The proposed method is based on excluding some classes and focusing on the most probable classes in the neighborhood space, named Local Crossing Off (LCO). This procedure is performed by employing a modified version of standard K-nearest neighbor and large margin nearest neighbor algorithms. The LCO method takes advantage of nearest neighbor classification algorithm because of its local learning behavior as well as the global behavior of powerful binary classifiers to discriminate between two classes. Combining these two properties in the proposed LCO technique will avoid the weaknesses of each method and will increase the efficiency of the whole classification system. On several benchmark datasets of varying size and difficulty, we found that the LCO approach leads to significant improvements using different base learners. The experimental results show that the proposed technique not only achieves better classification accuracy in comparison to other standard approaches, but also is computationally more efficient for tackling classification problems which have a relatively large number of target classes.
Similar content being viewed by others
Notes
From an ensemble classification point of view, the ECOC approach can be categorized in ensemble methods.
References
Allwein EL, Schapire RE, Singer Y (2001) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141
Anand R, Mehrotra K, Mohan CK, Ranka S (1995) Efficient classification for multiclass problems using modular neural networks. IEEE Trans Neural Netw 6(1):117–124
Blake C, Merz C (1998) Uci repository of machine learning databases, department of information and computer sciences. University of california, Irvine
Boiman O, Shechtman E, Irani M (2008) In defense of nearest-neighbor based image classification. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 1–8
Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27
Clark P, Boswell R (1991) Rule induction with cn2: Some recent improvements. In: Kodratoff Y (ed) Machine Learning-EWSL-91, vol 482. Lecture Notes in Computer Science. Springer, Berlin/Heidelberg, pp 151–163
Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
Dietterich T, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286
Escalera S, Pujol O, Radeva P (2010) On the decoding process in ternary error-correcting output codes. IEEE Trans Pattern Anal Mach Intell 32(1):120–134
Escalera S, Pujol O, Radeva P (2010) Re-coding ecocs without re-training. Pattern Recognit Lett 31:555–562
Fei B, Liu J (2006) Binary tree of svm: a new fast multiclass training and cassification algorithm. IEEE Trans Neural Netw 17:696–704
Fnrnkranz J (2002) Round robin classification. J Mach Learn Res 2:721–747
Garcfa-Pedrajas N, Ortiz-Boyer D (2011) An empirical study of binary classifier fusion methods for multiclass classification. Inform Fusion 12(2):111–130
Garcia-Pedrajas N, Ortiz-Boyer D (2006) Improving multiclass pattern recognition by the combination of two strategies. IEEE Trans Pattern Anal Mach Intell 28(6):1001–1006
Hastie T, Tibshirani R (1998) Classification by pairwise coupling. Ann Stat 26(2):451–471
Hnllermeier E, Vanderlooy S (2010) Combining predictions in pairwise classification: an optimal adaptive voting strategy and its relation to weighted voting. Pattern Recognit 43(1):128–142
Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425
Iman R, Davenport J (1980) Approximations of the critical regions of the friedman statistic. Commun Stat 6:571–595
Knerr S, Personnaz L, Dreyfus G (1990) Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Fogelman J (ed) Neurocomputing: algorithms, architectures and applications. Springer-Verlag, New York
Ko J, Byun H (2003) Binary classifier fusion based on the basic decomposition methods. In: Windeatt T, Roli F (eds) Multiple Classifier Systems, vol 2709., Lecture Notes in Computer ScienceSpringer, Berlin / Heidelberg, pp 159–159
Kumar S, Ghosh J, Crawford MM (2002) Hierarchical fusion of multiple classifiers for hyperspectral data analysis. Pattern Anal Appl 5(2):210–220
Nemenyi P (1963) Distribution-free multiple comparisons. Ph.D. thesis
Park SH, Fnrnkranz J (2007) Efficient pairwise classification. In: Kok J, Koronacki J, Mantaras R, Matwin S, Mladenic D, Skowron A (eds) Machine Learning: ECML 2007, vol 4701., Lecture Notes in Computer Science. Springer, Berlin/Heidelberg, pp 658–665
Platt J, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In: Solla S, Leen T, Mnller KR (eds) Proceedings of Neural Information Processing Systems, NIPS’99. MIT Press, London, pp 547–553
Pujol O, Radeva P, Vitria J (2006) Discriminant ecoc: a heuristic method for application dependent design of error correcting output codes. IEEE Trans Pattern Anal Mach Intell 28(6):1007–1012
Rifkin R, Klautau A (2004) In defense of one-vs-all classification. J Mach Learn Res 5:101–141
Vapnik V (1998) Statistical Learning Theory. Wiley, New York
Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244
Windeatt T, Ardeshir G (2001) An empirical comparison of pruning methods for ensemble classifiers
Wu T, Lin C, Weng R (2004) Probability estimates for multi-class classification by pairwise coupling. J Mach Learn Res 5:975–1005
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bagheri , M.A., Gao, Q. & Escalera, S. Combining local and global learners in the pairwise multiclass classification. Pattern Anal Applic 18, 845–860 (2015). https://doi.org/10.1007/s10044-014-0374-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-014-0374-x