Skip to main content

Advertisement

SpringerLink
Log in

Rule extraction algorithm from support vector machines and its application to credit screening

  • Original Paper
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Developing rule extraction algorithms from machine learning techniques such as artificial neural networks and support vector machines (SVMs), which are considered incomprehensible black-box models, is an important topic in current research. This study proposes a rule extraction algorithm from SVMs that uses a kernel-based clustering algorithm to integrate all support vectors and genetic algorithms into extracted rule sets. This study uses measurements of accuracy, sensitivity, specificity, coverage, fidelity and comprehensibility to evaluate the performance of the proposed method on the public credit screening data sets. Results indicate that the proposed method performs better than other rule extraction algorithms. Thus, the proposed algorithm is an essential analysis tool that can be effectively used in data mining fields.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Arbatli AD, Akin HL (1997) Rule extraction from trained neural networks using genetic algorithms. Nonlinear Anal 30(3):1639–1648. doi:10.1016/s0362-546x(96)00267-2

    Article  MATH  Google Scholar 

  • Barakat N, Bradley AP (2007) Rule extraction from support vector machines: a sequential covering approach. IEEE Trans Knowl Data Eng 19(6):729–741

    Article  Google Scholar 

  • Barakat N, Bradley AP (2010) Rule extraction from support vector machines: a review. Neurocomputing 74(1–3):178–190

    Article  Google Scholar 

  • Barakat N, Diederich J (2006) Eclectic rule-extraction from support vector machines. Int J Comput Intell 2(1):59–62

    Google Scholar 

  • Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum, New York

    MATH  Google Scholar 

  • Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167. doi:10.1023/a:1009715923555

    Article  Google Scholar 

  • Chaves AC, Vellasco M, Tanscheit R (2005) Fuzzy rule extraction from support vector machines. Paper presented at the Proceedings of the Fifth International Conference on Hybrid Intelligent Systems

  • Cohen WW (1995) Fast effective rule induction. In: Proceedings of the 12th International Conference on Maching Learning, pp 115–123

  • Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines. Cambridge University Press, Cambridge

    Google Scholar 

  • Davis LD, Mitchell M (1991) Handbook of genetic algorithms. Van Nostrand Reinhold, New York

  • Demsar J (2006) Statistical comparison of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  • Dhillon I, Guan Y, Kulis B (2005) A unified view of kernel k-means, spectral clustering and graph cuts. Technical Report TR-04-25. University of Texas at Austin

  • Diederich J (2008) Rule extraction from support vector machines: an introduction. In: Rule extraction from support vector machines, vol 80. Studies in Computational Intelligence. Springer, Berlin, pp 3–31. doi:10.1007/978-3-540-75390-2_1

  • Ebrahim Elalfi A, Haque R, Elalami ME (2004) Extracting rules from trained neural network using GA for managing E-business. Appl Soft Comput 4:65–77

    Article  Google Scholar 

  • Frank A, Asuncion A (2010) UCI Machine Learning Repository. http://archive.ics.uci.edu/ml

  • Fung G, Sandilya S, Rao RB (2005) Rule extraction from linear support vector machines. In: Proceedings of the eleventh ACM SIGKDD international conference on knowledge discovery in data mining, Chicago. doi:http://doi.acm.org/10.1145/1081870.1081878

  • García-Martínez C, Lozano M (2010) Evaluating a local genetic algorithm as context-independent local search operator for metaheuristics. Soft Comput 14(10):1117–1139. doi:10.1007/s00500-009-0506-1

    Article  Google Scholar 

  • Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley Longman Publishing Co., Inc, Boston

  • Hartigan JA, Wong MA (1979) A K-means clustering algorithm. Appl Stat 28:100–108

    Article  MATH  Google Scholar 

  • Holland J (1975) Adaptation in natural and artificial systems. The University of Michigan Press, Ann Arbor

    Google Scholar 

  • Martens D, Baesens B, Van Gestel T, Vanthienen J (2007) Comprehensible credit scoring models using rule extraction from support vector machines. Eur J Oper Res 183(3):1466–1476

    Article  MATH  Google Scholar 

  • Martens D, Huysmans J, Setiono R, Vanthienen J, Baesens B (2008) Rule extraction from support vector machines: an overview of issues and application in credit scoring. Stud Comput Intell 80:33–63

    Article  Google Scholar 

  • Martens D, Baesens B, Gestel TV (2009a) Decompositional rule extraction from support vector machines by active learning. IEEE Trans Knowl Data Eng 21:177–190

    Article  Google Scholar 

  • Martens D, Baesens BB, Van Gestel T (2009b) Decompositional rule extraction from support vector machines by active learning. IEEE Trans Knowl Data Eng 21(2):178–191

    Article  Google Scholar 

  • Michalewicz Z (1996) Genetic algorithms + Data Structures = Evolution Programs. Springer, Berlin

  • Núñez H, Angulo C, Catala A (2002) Rule extraction from support vector machines. In: Proceedings of European Symposium on Artificial Neural Networks, pp 107–112

  • Núñez H, Angulo C, Catala A (2006) Rule-based learning systems for support vector machines. Neural Process Lett 24(1):1–18. doi:http://dx.doi.org/10.1007/s11063-006-9007-8

  • Prechelt L (1994) Proben1—a set of neural network benchmark problems and benchmarking rules. Technical report 21/94, Fakultät für Informatik, Universität Karlsruhe, Germany, Anonymous ftp

  • Quinlan JR (1993) Programs for machine learning. Morgan Kaufmann, San Francisco

  • Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):23–38

    Article  Google Scholar 

  • Schlkopf B, Smola AJ (2002) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge

  • Setiono R, Baesens B, Mues C (2009) A note on knowledge discovery using neural networks and its application to credit card screening. Eur J Oper Res 192(1):326–332

    Article  MATH  Google Scholar 

  • Sexton RS, McMurtrey S, Cleavenger D (2006) Knowledge discovery using a neural network simultaneous optimization algorithm on a real world classification problem. Eur J Oper Res 168(3):1009–1018

    Article  MATH  Google Scholar 

  • Shen Y, Li T, Hermans E, Ruan D, Wets G, Vanhoof K, Brijs T (2010) A hybrid system of neural networks and rough sets for road safety performance indicators. Soft Comput 14(12):1255–1263. doi:10.1007/s00500-009-0492-3

    Article  Google Scholar 

  • Silva-Ramirez EL, Pino-Mejias R, Lopez-Coello M, Cubiles-de-la-Vega MD (2011) Missing value imputation on missing completely at random data using multilayer perceptrons. Neural Netw 24:121–129

    Article  Google Scholar 

  • Tan P-N, Steinbach M, Kumar V (2005) Introduction to data mining. Addison Wesley, Boston

  • Tan F, Fu X, Zhang Y, Bourgeois AG (2008) A genetic algorithm-based method for feature subset selection. Soft Comput 12(2):111–120. doi:10.1007/s00500-007-0193-8

    Google Scholar 

Download references

Acknowledgment

This work has been partially supported by grants NSC-100-2410-H-007-050-MY2 from National Science Council, Taiwan.

Author information

Authors and Affiliations

  1. Department of Industrial Engineering and Engineering Management, National Tsing Hua University, 101, Sec. 2, Kuang Fu Road, Hsinchu, 300, Taiwan

    Chao-Ton Su & Yan-Cheng Chen

Authors
  1. Chao-Ton Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan-Cheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao-Ton Su.

Appendix A: The pseudo code for KCGex-SVM

Appendix A: The pseudo code for KCGex-SVM

Rights and permissions

Reprints and permissions

About this article

Cite this article

Su, CT., Chen, YC. Rule extraction algorithm from support vector machines and its application to credit screening. Soft Comput 16, 645–658 (2012). https://doi.org/10.1007/s00500-011-0762-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-011-0762-8

Keywords

Search

Navigation