Abstract
In this paper, we propose using dynamic ensemble selection (DES) method on ensemble generated based on random projection. We first construct the homogeneous ensemble in which a set of base classifier is obtained by a learning algorithm on different training schemes generated by projecting the original training set to lower dimensional down spaces. We then develop a DES method on those base classifiers so that a subset of base classifiers is selected to predict label for each test sample. Here competence of a classifier is evaluated based on its prediction results on the test sample’s \( k - \) nearest neighbors obtaining from the projected data of validation set. Our proposed method, therefore, gains the benefits not only from the random projection in dimensionality reduction and diverse training schemes generation but also from DES method in choosing an appropriate subset of base classifiers for each test sample. The experiments conducted on some datasets selected from four different sources indicate that our framework is better than many state-of-the-art DES methods concerning to classification accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Britto, A.S., Sabourin, R., Oliveira, L.E.S.: Dynamic selection of classifiers—a comprehensive review. Pattern Recog. 47(11), 3665–3680 (2014)
Breiman, L.: Bagging predictors. Mach. Learn. 24, 123–140 (1996)
Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20(8), 832–844 (1998)
Johnson, W., Lindenstrauss, J.: Extensions of Lipschitz mapping into Hilbert space. In: Conference in Modern Analysis and Probability, vol. 26, pp. 189–206 (1984). Contemporary Mathematics, American Mathematical Society
Fern, X.Z., Brodley, C.E.: Random projection for high dimensional data clustering: a cluster ensemble approach. In: ICML, pp. 186–193 (2003)
Cruz, R.M.O., Sabourin, R., Cavalcanti, G.D.C., Ren, T.I.: META-DES: a dynamic ensemble selection framework using meta-learning. Pattern Recog. 48(5), 1925–1935 (2015)
Nguyen, T.T., Nguyen, T.T.T., Pham, X.C., Liew, A.W.-C., Bezdek, J.C.: An ensemble-based online learning algorithm for streaming data. CoRR abs/1704.07938 (2017)
Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: ACM SIGKDD, pp. 245–250 (2001)
Wang, Z., Yuan, X.-T., Liu, Q.: Sparse random projection for χ2 kernel linearization: algorithm and applications to image classification. Neurocomputing 151(1, 3), 327–332 (2015)
Pham, X.C., Dang, M.T., Dinh, V.S., Hoang, S., Nguyen, T.T., Liew, A.W.-C.: Learning from data stream based on random projection and Hoeffding tree classifier. In: DICTA 2017 (in press)
Rathore, P., Bezdek, J.C., Erfani, S.M., Rajasegarar, S., Palaniswami, M.: Ensemble fuzzy clustering using cumulative aggregation on random projections. IEEE Trans. Fuzzy Syst. (2017, in press). https://doi.org/10.1109/tfuzz.2017.2729501
Cruz, R.M.O., Sabourin, R., Cavalcanti, G.D.C.: Dynamic classifier selection: recent advances and perspectives. Inf. Fusion. 41, 195–216 (2018)
Giacinto, G., Roli, F.: Dynamic classifier selection based on multiple classifier behaviour. Pattern Recogn. 34, 1879–1881 (2001)
Smits, P.C.: Multiple classifier systems for supervised remote sensing image classification based on dynamic classifier selection. IEEE Trans. Geosci. Remote Sens. 40(4), 801–813 (2002)
Cavalin, P.R., Sabourin, R., Suen, C.Y.: Dynamic selection approaches for multiple classifier systems. Neural Comput. Appl. 22(3–4), 673–688 (2013)
Ko, A.H.R., Sabourin, R., Britto, A.S.: From dynamic classifier selection to dynamic ensemble selection. Pattern Recogn. 41, 1735–1748 (2008)
Cruz, R.M.O., Cavalcanti, G.D.C., Ren, T.I.: A method for dynamic ensemble selection based on a filter and an adaptive distance to improve the quality of the regions of competence. In: IJCNN, pp. 1126–1133 (2011)
Soares, R.G.F., Santana, A., Canuto, A.M.P., de Souto, M.C.P.: Using accuracy and diversity to select classifiers to build ensembles. In: IJCNN, pp. 1310–1316 (2006)
Woloszynski, T., Kurzynski, M.: A probabilistic model of classifier competence for dynamic ensemble selection. Pattern Recogn. 44, 2656–2668 (2011)
Woloszynski, T., Kurzynski, M., Podsiadlo, P., Stachowiak, G.W.: A measure of competence based on random classification for dynamic ensemble selection. Inf. Fusion 13(3), 207–213 (2012)
Nguyen, T.T., Nguyen, T.T.T., Pham, X.C., Liew, A.W.-C.: A novel combining classifier method based on variational inference. Pattern Recogn. 49, 198–212 (2016)
Nguyen, T.T., Nguyen, M.P., Pham, X.C., Liew, A.W.-C.: A hybrid classification system with fuzzy rule and classifier ensemble. Inf. Sci. 422, 144–160 (2018)
Nguyen, T.T., Liew, A.W.-C., Pham, X.C., Nguyen, M.P.: A novel 2-stage combining classifier model with stacking and genetic algorithm based feature selection. In: Huang, D.-S., Jo, K.-H., Wang, L. (eds.) ICIC 2014. LNCS (LNAI), vol. 8589, pp. 33–43. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09339-0_4
Nguyen, T.T., Liew, A.W.-C., Tran, M.T., Nguyen, M.P.: Combining multi classifiers based on a genetic algorithm – a Gaussian mixture model framework. In: Huang, D.-S., Jo, K.-H., Wang, L. (eds.) ICIC 2014. LNCS (LNAI), vol. 8589, pp. 56–67. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09339-0_6
Nguyen, T.T., Liew, A.W.-C., Tran, M.T., Pham, X.C., Nguyen, M.P.: A novel genetic algorithm approach for simultaneous feature and classifier selection in multi classifier system. In: CEC, pp. 1698–1705 (2014)
Nguyen, T.T., Pham, X.C., Liew, A.W.-C., Pedrycz, W.: Aggregation of classifiers: a justifiable information granularity approach. CoRR abs/1703.05411 (2017)
Nguyen, T.T., Pham, X.C., Liew, A.W.-C., Nguyen, M.P.: Optimization of ensemble classifier system based on multiple objectives genetic algorithm. In: ICMLC, vol. 1, pp. 46–51 (2014)
Bache, K., Lichman, M.: UCI Machine Learning Repository (2013)
King, R.D., Feng, C., Sutherland, A.: STATLOG: comparison of classification algorithms on large real-world problems. Appl. Artif. Intell. Int. J. 9(3), 289–333 (1995)
Alcalá-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. Mult. Val. Log. Soft Comput. 17(2–3), 255–287 (2011)
Kuncheva, L.: Ludmila Kuncheva collection LKC (2004)
Garcia, S., Herrera, F.: An extension on statistical comparisons of classifiers over multiple data sets for all pairwise comparisons. J. Mach. Learn. Res. 9, 2579–2596 (2008)
Nguyen, T.T., Weidlich, M., Duong, C.T., Yin, H., Nguyen, Q.V.H.: Retaining data from streams of social platforms with minimal regret. In: IJCAI, pp. 2850–2856 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Dang, M.T., Luong, A.V., Vu, TT., Nguyen, Q.V.H., Nguyen, T.T., Stantic, B. (2018). An Ensemble System with Random Projection and Dynamic Ensemble Selection. In: Nguyen, N., Hoang, D., Hong, TP., Pham, H., Trawiński, B. (eds) Intelligent Information and Database Systems. ACIIDS 2018. Lecture Notes in Computer Science(), vol 10751. Springer, Cham. https://doi.org/10.1007/978-3-319-75417-8_54
Download citation
DOI: https://doi.org/10.1007/978-3-319-75417-8_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-75416-1
Online ISBN: 978-3-319-75417-8
eBook Packages: Computer ScienceComputer Science (R0)