Abstract
Due to the rapid growth of computer technologies and the extensive changes in human needs, expertise and digital information were used to induce general conclusions. Such conclusions can be used to deal with future activates and make the life of humans easier. One active filed of machine learning that was developed for this purpose is inductive learning, and several families have emerged from this field. Specifically, RULES family was discovered as covering algorithm that directly induces good and general conclusions in the shape of simple rules. However, it was found that RULES suffer from two major deficiencies. It needs to tradeoff between time and accuracy when inducing the best rule and it did not appropriately handle incomplete data. As a result, this paper will present a new RULES algorithm, which takes advantage of previous versions of RULES family in addition to other advance methods of machine learning, specifically Transfer learning. Moreover, multi-modeling is also merged to transfer the knowledge of a different classification model and further improve the original algorithm. At the end, an empirical test is applied to compare the proposed algorithm with different single-model algorithms to prove that using the past knowledge of other agents in different domains improves specialization accuracy, whether the data is complete or incomplete.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)
Stahl, F., Bramer, M.: Computationally efficient induction of classification rules with the PMCRI and J-PMCRI frameworks. Knowledge-Based Systems (2012)
Kurgan, L.A., Cios, K.J., Dick, S.: Highly Scalable and Robust Rule Learner: Performance Evaluation and Comparison. IEEE Systems, Man, and Cybernetics—Part B: Cybernetics 36, 32–53 (2006)
Aksoy, M.: A review of rules family of algorithms. Mathematical and Computational Applications 13, 51–60 (2008)
Theron, H.: An Empirical Evaluation of Beam Search and Pruning in BEXA. In: Fifth International Conference on Tools with Artificial Intelligence (TAI 1993), Boston, MA, pp. 132–139 (1993)
Kotsiantis, S.B.: Supervised Machine Learning: A Review of Classification Techniques. Presented at the Proceedings of the 2007 Conference on Emerging Artificial Intelligence Applications in Computer Engineering: Real Word AI Systems with Applications in eHealth, HCI, Information Retrieval and Pervasive Technologies (2007)
Maclin, R., Opitz, D.W.: Popular Ensemble Methods: An Empirical Study. Journal of Artificial Intelligence Research 11, 169–198 (1999)
Pham, D., Afify, A.: Machine-learning techniques and their applications in manufacturing. Proceedings of the I MECH E Part B Journal of Engineering Manufacture 219, 395–412 (2005)
ElGibreen, H., Aksoy, M.: RULES – TL: A simple and Improved RULES Algorithm for Incomplete and Large Data. Journal of Theoretical and Applied Information Technology 47 (2013)
Sinno Jialin, P., Qiang, Y.: A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering 22, 1345–1359 (2010)
Pham, D., Bigot, S., Dimov, S.: RULES-5: a rule induction algorithm for classification problems involving continuous attributes. In: Institution of Mechanical Engineers, pp. 1273–1286 (2003)
Ramon, J., Driessens, K., Croonenborghs, T.: Transfer Learning in Reinforcement Learning Problems Through Partial Policy Recycling. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 699–707. Springer, Heidelberg (2007)
Taylor, M., Suay, H.B., Chernova, S.: Integrating Reinforcement Learning with Human Demonstrations of Varying Ability. In: International Conferance of Autonomous Agents and Multiagent Systems (AAMAS 2011), Taipei, Taiwan (2011)
Mahmud, M.M.H.: On Universal Transfer Learning. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) ALT 2007. LNCS (LNAI), vol. 4754, pp. 135–149. Springer, Heidelberg (2007)
Yongqiang, L.: A Review About Transfer Learning Methods and Applications. In: International Conference on Information and Network Technology IPCSIT, Singapore, pp. 7–11 (2011)
Pham, D.T., Aksoy, M.S.: RULES: A simple rule extraction system. Expert Systems with Applications 8, 59–65 (1995)
Pham, D.T., Aksoy, M.S.: An algorithm for automatic rule induction. Artificial Intelligence in Engineering 8, 277–282 (1993)
Pham, D.T., Aksoy, M.S.: A new algorithm for inductive learning. Journal of Systems Engenering 5, 115–122 (1995)
Pham, D.T., Dimov, S.S.: The RULES-3 Plus inductive learning algorithm. In: Proceedings of the Third World Congress on Expert Systems, Seoul, Korea, pp. 917–924 (1996)
Aksoy, M.S., Mathkour, H.: CSLS: Connectionist Symbolic Learning System. Mathematical and Computational Applications 14, 177–186 (2009)
Pham, D.T., Dimov, S.S.: An algorithm for incremental inductive learning. Journal of Engineering Manufacture 211, 239–249 (1997)
Pham, D.T., Soroka, A.J.: An Immune-network inspired rule generation algorithm (RULES-IS). In: Third Virtual International Conference on Innovative Production Machines and Systems. Whittles, Dunbeath (2007)
Pham, D.T., Bigot, S., Dimov, S.S.: RULES-F: A fuzzy inductive learning algorithm. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 220, 1433–1447 (2006)
Bigot, S.: A new rule space representation scheme for rule induction in classification and control applications. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering (2011)
Pham, D.T., Afify, A.A.: RULES-6: A Simple Rule Induction Algorithm for Supporting Decision Making. Presented at the 31st Annual Conference of IEEE Industrial Electronics Society (IECON 2005) (2005)
Shehzad, K.: EDISC: A Class-tailored Discretization Technique for Rule-based Classification. IEEE Transactions on Knowledge and Data Engineering 24, 1435–1447 (2012)
Afify, A.A., Pham, D.T.: SRI: A Scalable Rule Induction Algorithm. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 220, 537–552 (2006)
Pan, W., Zhong, E., Yang, Q.: Transfer Learning for Text Mining, pp. 223–257 (2012)
Xie, Y.-F., Su, S.-Z., Li, S.-Z.: A Pedestrian Classification Method Based on Transfer Learning. Presented at the International Conference on Image Analysis and Signal Processing, IASP, Zhejiang (2010)
Estévez, J.I., Toledo, P.A., Alayón, S.: Using an Induced Relational Decision Tree for Rule Injection in a Learning Classifier System. Presented at the IEEE Congress on Evolutionary Computation New Orleans, LA (2011)
Boström, H.: Induction of Recursive Transfer Rules. In: Cussens, J., Džeroski, S. (eds.) LLL 1999. LNCS (LNAI), vol. 1925, pp. 237–450. Springer, Heidelberg (2000)
Reid, M.D.: DEFT Guessing: Using Inductive Transfer to Improve Rule Evaluation from Limited Data. In: Doctor of Philosophy, School of Computer Science and Engineering. University of New South Wales, Sydney (2007)
Ganchev, P., Malehorn, D., Bigbee, W.L., Gopalakrishnan, V.: Transfer learning of classification rules for biomarker discovery and verification from molecular profiling studies. Journal of Biomedical Informatics 44(suppl. 1), S17–S23 (2011)
Bigot, S.: A study of specialisation and classification heuristics used in covering algorithms. Presented at the IPROM2009 Innovative Production Machines and Systems Fifth I*PROMS Virtual Conference, Cardiff, UK (2009)
Alcalá-Fdez, J., Sánchez, L., García, S., del Jesus, M.J., Ventura, S., Garrell, J.M., Otero, J., Romero, C., Bacardit, J., Rivas, V.M., Fernández, J.C., Herrera, F.: KEEL: A Software Tool to Assess Evolutionary Algorithms to Data Mining Problems. Soft Computing 13, 307–318 (2009)
Alcalá-Fdez, J., Fernandez, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework. Journal of Multiple-Valued Logic and Soft Computing 17, 255–287 (2011)
Efron, B., Tibshirani, R.: An Introduction to the Bootstrap. Chapman & Hall, USA (1993)
Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuousvalued attributes for classification learning. Presented at the 13th International Joint Conference of Artificial Intelligence (1993)
Cai, Z.: Technical Aspects of Data Mining. PhD, University of Wales Cardiff, Cardiff, UK (2001)
Pham, D.T., Afify, A.A.: Online Discretization of Continuous-Valued Attributes in Rule Induction. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 219, 829–842 (2005)
Luengo, J., García, S., Herrera, F.: On the choice of the best imputation methods for missing values considering three groups of classification methods. Knowledge and Information Systems 32, 77–108 (2012)
Li, D., Deogun, J., Spaulding, W., Shuart, B.: Towards Missing Data Imputation: A Study of Fuzzy K-means Clustering Method. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 573–579. Springer, Heidelberg (2004)
Cohen, W.W.: Fast Effective Rule Induction. In: Twelfth International Conference on Machine Learning, pp. 115–123 (1995)
Stahl, F., Bramer, M., Adda, M.: P-Prism: A Computationally Efficient Approach to Scaling up Classification Rule Induction. In: Bramer, M. (ed.) Artificial Intelligence in Theory and Practice II. IFIP, vol. 276, pp. 77–86. Springer, Heidelberg (2008)
Stahl, F., Bramer, M.: Induction of Modular Classification Rules: Using Jmax-pruning, pp. 79–92 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
ElGibreen, H., Aksoy, M.S. (2013). Multi Model Transfer Learning with RULES Family. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2013. Lecture Notes in Computer Science(), vol 7988. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39712-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-39712-7_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39711-0
Online ISBN: 978-3-642-39712-7
eBook Packages: Computer ScienceComputer Science (R0)