Abstract
In this paper, we carefully review all our work since 2012 and establish the concept of the Multiple-MLP Ensemble Re-RX algorithm. We first examine the background and procedures of the Recursive-Rule Extraction (Re-RX) algorithm family and its variants, including the Multiple-MLP Ensemble Re-RX algorithm (“Multiple-MLP Ensemble”), which uses the Re-RX algorithm as its core to find the rules for seven kinds of mixed (i.e., discrete and continuous attributes) datasets. We compare the accuracy against only the Re-RX algorithm family. The Multiple-MLP Ensemble Re-RX algorithm cascades standard backpropagation (BP) to train a multiple neural-network ensemble, where each neural network is a Multi-Layer Perceptron (MLP). Strictly speaking, multiple neural networks do not need to be trained simultaneously. Therefore, the Multiple-MLP Ensemble avoids the previous complicated neural network ensemble structures and the difficulties of rule extraction algorithms. The extremely high accuracy of the Multiple-MLP Ensemble algorithm generally outperformed the Re-RX algorithm and its variant. The results confirm that the Multiple-MLP Ensemble approach facilitates the migration from existing data systems toward new analytic systems and Big data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Setiono, R., et al.: Recursive neural network rule extraction for data with mixed attributes. IEEE Trans. Neural Netw. 19(2), 299–307 (2008)
Setiono, R., et al.: Rule extraction from minimal neural networks for credit card screening. Inter. J. of Neural Systems 21(4), 265–276 (2011)
Setiono, R., et al.: A note on knowledge discovery using neural networks and its application to credit card screening. European J. Operational Research 192, 326–332 (2009)
Hayashi, Y., et al.: Understanding consumer heterogeneity: A business intelligence application of neural networks. Knowledge-Based Systems 23(8), 856–863 (2010)
Bologna, G.: Is it worth generating rules from neural network ensemble? J. of Applied Logic 2, 325–348 (2004)
Zhou, Z.-H.: Extracting symbolic rules from trained neural network ensembles. AI Communications 16, 3–15 (2003)
Hara, A., Hayashi, Y.: Ensemble neural network rule extraction using Re-RX algorithm. In: Proc. WCCI (IJCNN 2012), Brisbane, Australia, June 10-15, pp. 604–609 (2012)
Hara, A., Hayashi, Y.: A New Neural Data Analysis Approach Using Ensemble Neural Network Rule Extraction. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds.) ICANN 2012, Part I. LNCS, vol. 7552, pp. 515–522. Springer, Heidelberg (2012)
Hayashi, Y., et al.: A New approach to Three Ensemble neural network rule extraction using Recursive-Rule eXtraction algorithm. In: Int. Joint Conf. Neural Networks (IJCNN 2013), Dallas, August 4-9, pp. 835–841 (2013)
Hayashi, Y., Tanaka, Y., Fujisawa, S., Izawa, T.: Comparative Study of Accuracies on the Family of the Recursive Rule Extraction Algorithm. In: Wermter, S., Weber, C., Duch, W., Honkela, T., Koprinkova-Hristova, P., Magg, S., Palm, G., Villa, A.E.P. (eds.) ICANN 2014. LNCS, vol. 8681, pp. 491–498. Springer, Heidelberg (2014)
Hayashi, Y.: Neural network rule extraction by a new ensemble concept and its theoretical and historical background: A review. Int. J. of Computational Intelligence and Applications 12(4), 1340006-1–1340006-22 (2013)
Setiono, R., et al.: A penalty-function approach for pruning feedforward neural networks. Neural Comp. 9(1), 185–204 (1997)
Univ. of California, Irvine Learning Repository, http://archive.ics.uci.edu/m/
Baesens, B., et al.: Using neural network rule extraction and decision tables for credit-risk evaluation. Management Science 49(3), 312–329 (2004)
Quinlan, R.: C4.5: Programming for Machine Learning. Morgan Kaufman, San Mateo (1993)
Duch, W., et al.: A New Methodology of Extraction, Optimization and Application of Crisp and Fuzzy Logical Rules. IEEE Trans. Neural Networks 12, 277–306 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Hayashi, Y., Tanaka, Y., Izawa, T., Fujisawa, S. (2014). Characteristics and Potential Developments of Multiple-MLP Ensemble Re-RX Algorithm. In: Loo, C.K., Yap, K.S., Wong, K.W., Teoh, A., Huang, K. (eds) Neural Information Processing. ICONIP 2014. Lecture Notes in Computer Science, vol 8834. Springer, Cham. https://doi.org/10.1007/978-3-319-12637-1_77
Download citation
DOI: https://doi.org/10.1007/978-3-319-12637-1_77
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12636-4
Online ISBN: 978-3-319-12637-1
eBook Packages: Computer ScienceComputer Science (R0)