Skip to main content
SpringerLink
Log in

Rule Extraction from Support Vector Machine Using Modified Active Learning Based Approach: An Application to CRM

  • Conference paper
Knowledge-Based and Intelligent Information and Engineering Systems (KES 2010)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 6276))

Abstract

Despite superior generalization performance Support vector machines (SVMs) generate black box models. The process of converting such opaque models into transparent model is often regarded as rule extraction. This paper presents a new approach for rule extraction from SVMs using modified active learning based approach (mALBA), to predict churn in bank credit cards. The dataset is obtained from Business Intelligence Cup 2004, which is highly unbalanced with 93% loyal and 7% churned customers’ data. Since identifying churner is paramount from business perspective, therefore considering sensitivity alone, the empirical results suggest that the proposed rule extraction approach using mALBA yielded the best sensitivity compared to other classifiers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Usama, F., Piatetsky-Shapiro, G., Smyth, P.: From Data Mining to Knowledge Discovery in Databases. American Association for Artificial Intelligence (1996)

    Google Scholar 

  2. Adriaans, P., Zantinge, D.: Data Mining. Addison-Wesley, New York (1996)

    Google Scholar 

  3. Osuna, E.E., Freund, R., Girosi, F.: Support vector Machines: Training and Applications. Technical Report, Massachusetts Institute of Technology, Artificial Intelligence Laboratory, AI Memo No. 1602 (1997)

    Google Scholar 

  4. Osuna, E.E., Freund, R., Girosi, F.: Training support vector machines: an application to face detection. In: Proceedings of Computer Vision and Pattern Recognition, pp. 130–136 (1997)

    Google Scholar 

  5. Senator, T., Goldberg, H.G., Wooton, J., Cottini, M.A., Umarkhan, A.F., Klinger, C.D., Llamas, W.M., Marrone, M.P., Wong, R.W.H.: The Financial Crimes Enforcement Network AI System (FAIS): Identifying Potential Money Laundering from Reports of Large Cash Transactions. AI Magazine 16(4), 21–39 (1995)

    Google Scholar 

  6. Hall, J., Mani, G., Barr, D.: Applying Computational Intelligence to the Investment Process. In: Proceedings of CIFER 1996: Computational Intelligence in Financial Engineering, Washington, D.C. IEEE Computer Society Press, Los Alamitos (1996)

    Google Scholar 

  7. Ravi, V., Arul Shalom, S.A., Manickavel, A.: Sputter Process Variables Prediction via Data Mining. In: Proceedings of the 2004 IEEE Conference on Cybernetics and Intelligent Systems, Singapore (2004)

    Google Scholar 

  8. Kumar, D.A., Ravi, V.: Predicting credit card customer churn in banks using data mining. International Journal for Data Analysis, Techniques and Strategies 1(1), 4–28 (2008)

    Google Scholar 

  9. Bolton, R.N., Kannan, P.K., Bramlett, M.D.: Implications of loyalty program membership and service experiences for customer retention and value. Journal of the Academy of Marketing Science 28(1), 95–108 (2000)

    Article  Google Scholar 

  10. Lejeune, M.A.P.M.: Measuring the impact of data mining on churn management. Electronic Networking Applications and Policy 11(5), 375–387 (2001)

    Article  Google Scholar 

  11. Mols, N.P.: The Behavioral consequences of PC banking. International Journal of Bank Marketing 16(5), 195–201 (1998)

    Article  Google Scholar 

  12. Chu, B.H., Tsai, M.-S., Ho, C.-S.: Toward a hybrid data mining model for customer retention. Knowledge-Based Systems 20(8), 703–718 (2007)

    Article  Google Scholar 

  13. Ryals, L., Knox, S.: Cross-functional issues in the implementation of relationship marketing through customer relationship management. European Management Journal 19(5), 534–542 (2001)

    Article  Google Scholar 

  14. Alajoutsijarvi, K., Mannermaaa, K., Tikkanen, H.: Customer relationships and the small software firm: a framework for understanding challenges faced in marketing. Information and Management 37, 153–159 (2000)

    Article  Google Scholar 

  15. Laudon, K., Laudon, J.: Management Information Systems: Managing the Digital Firm, 7th edn. Prentice-Hall, Englewood Cliffs (2000)

    Google Scholar 

  16. Edwards, J.: Get It Together with Collaborative CRM. insideCRM. Tippit (2007), http://www.insidecrm.com/features/collaborative-crm-112907/

  17. Bloemer, J., Ruyter, K.D., Peeters, P.: Investigating drivers of bank loyalty: the complex relationship between image, service quality and satisfaction. International Journal of Bank Marketing 16(7), 276–286 (1998)

    Article  Google Scholar 

  18. Mozer, M.C., Wolniewicz, R., Grimes, D.B., Johnson, E., Kaushansky, H.: Predicting subscriber dissatisfaction and improving retention in the wireless telecommunications industry. IEEE Transactions of Neural Networks 11(3), 690–696 (2000)

    Article  Google Scholar 

  19. Larivière, B., Van den Poel, D.: Investigating the role of product features in preventing customer churn, by using survival analysis and choice modeling: the case of financial services. Expert Systems with Applications 27(2), 277–285 (2004)

    Article  Google Scholar 

  20. Hung, S.-Y., Yen, D.C., Wang, H.: Applying data mining to telecom churn management. Expert Systems with Applications 31(3), 515–524 (2006)

    Article  Google Scholar 

  21. Naveen, N., Ravi, V., Kumar, D.A.: Application of fuzzyARTMAP for churn prediction in bank credit cards. International Journal of Information and Decision Sciences 1(4), 428–444 (2009)

    Article  Google Scholar 

  22. Farquad, M.A.H., Ravi, V., Bapi, R.S.: Data Mining using Rules Extracted from SVM: an Application to Churn Prediction in Bank Credit Cards. In: Sakai, H., Chakraborty, M.K., Hassanien, A.E., Ślęzak, D., Zhu, W. (eds.) RSFDGrC 2009. LNCS, vol. 5908, pp. 390–397. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  23. Zhao, Y., Li, B., Li, X., Liu, W., Ren, S.: Customer Churn Prediction Using Improved One-Class Support Vector Machine. In: Li, X., Wang, S., Dong, Z.Y. (eds.) ADMA 2005. LNCS (LNAI), vol. 3584, pp. 300–306. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  24. Coussement, K., Van den Poel, D.: Churn prediction in subscription services: An application of support vector machines while comparing two parameter-selection techniques. Expert Systems with Applications 34(1), 313–327 (2008)

    Article  Google Scholar 

  25. Zhaoa., J., Dang, X.-H.: Bank Customer Churn Prediction Based on Support Vector Machine: Taking a Commercial Bank’s VIP Customer Churn as the Example. In: Proceedings of the 4th International Conference on Wireless Communications, Networking and Mobile Computing 2008 (WiCOM 2008), pp. 1–4 (2008)

    Google Scholar 

  26. Cao., K., Shao, P.-j.: Customer Churn Prediction Based on SVM-RFE. In: International Seminar on Business and Information Management ISBIM 2008, vol. 1, pp. 306–309 (2008)

    Google Scholar 

  27. Huang, B.Q., et al.: A new feature set with new window techniques for customer churn prediction in land-line telecommunications. Expert Systems with Applications 37(5), 3657–3665 (2010)

    Article  Google Scholar 

  28. Gallant, S.: Connectionist expert systems. Communications of the ACM 31(2), 152–169 (1988)

    Article  Google Scholar 

  29. Davis, R., Buchanan, B.G., Shortliffe, E.: Production rules as a representation for a knowledge-based consultation program. Artificial Intelligence 8(1), 15–45 (1977)

    Article  MATH  Google Scholar 

  30. Craven, M.W., Shavlik, J.W.: Using sampling and queries to extract rules from trained neural networks. In: The Proceedings of the Eleventh International Conference on Machine Learning, San Francisco, CA, USA (1994)

    Google Scholar 

  31. Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)

    MATH  Google Scholar 

  32. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.N.: Gene selection for cancer classification using support vector machines. Machine Learning 46(1-3), 389–422 (2002)

    Article  MATH  Google Scholar 

  33. Tickle, A.B., Andrews, R., Golea, M., Diederich, J.: The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural network. IEEE Transactions on Neural Networks 9(6), 1057–1068 (1998)

    Article  Google Scholar 

  34. Nunez, H., Angulo, C., Catata, A.: Rule extraction from support vector machines. In: European Symposium on Artificial Neural Networks Proceedings, pp. 107–112 (2002)

    Google Scholar 

  35. Fung, G., Sandilya, S., Bharat, R.R.: Rule extraction from linear support vector machines. In: Proceeding of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, pp. 32–40. ACM Press, New York (2005)

    Chapter  Google Scholar 

  36. Barakath, N., Diederich, J.: Eclectic rule-extraction from support vector machines. International journal of Computer Intelligence 2(1), 59–62 (2005)

    Google Scholar 

  37. Zhang, Y., Su, H., Jia, T., Chu, J.: Rule Extraction from Trained Support Vector Machines. In: Ho, T.-B., Cheung, D., Liu, H. (eds.) PAKDD 2005. LNCS (LNAI), vol. 3518, pp. 61–70. Springer, Heidelberg (2005)

    Google Scholar 

  38. Chaves, A.C.F., Vellasco, M.M.B.R., Tanscheit, R.: Fuzzy rule extraction from support vector machines. In: Fifth International Conference on Hybrid Intelligent Systems, Rio de Janeiro, Brazil, November 06-09 (2005)

    Google Scholar 

  39. Chen, Z., Li, J., Wei, L.: A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue. Artificial Intelligence in Medicine 41, 161–175 (2007)

    Article  Google Scholar 

  40. Barakat, N.H., Bradley, A.P.: Rule Extraction from Support Vector Machines: A Sequential Covering Approach. IEEE Transactions on Knowledge and Data Engineering 19(6), 729–741 (2007)

    Article  Google Scholar 

  41. Martens, D., Baesens, B., Gestel, T.V.: Decompositional Rule Extraction from Support Vector Machines by Active Learning. IEEE Transactions on Knowledge and Data Engineering 21(2), 178–191 (2009)

    Article  Google Scholar 

  42. Farquad, M.A.H., Ravi, V., Bapi, R.S.: Rule Extraction using Support Vector Machine Based Hybrid Classifier. In: Presented in TENCON-2008, IEEE region 10 Conference, Hyderabad, India, November 19-21 (2008)

    Google Scholar 

  43. Farquad, M.A.H., Ravi, V., Bapi, R.S.: Support Vector Machine based Hybrid Classifiers and Rule Extraction Thereof: Application to Bankruptcy Prediction in Banks. In: Soria, E., Martín, J.D., Magdalena, R., Martínez, M., Serrano, A.J. (eds.) Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods and Techniques, vol. 2, pp. 404–426. IGI Global, USA (2010)

    Google Scholar 

  44. Farquad, M.A.H., Ravi, V., Bapi, R.S.: Support vector regression based hybrid rule extraction methods for forecasting. Expert Systems with Applications (2010), doi:10.1016/j.eswa.2010.02.055

    Google Scholar 

  45. Ron, K.: Scaling Up the Accuracy of Naïve-Bayes Classifiers: a Decisoin-Tree Hybrid. In: Proceedings of KDD 1996, Portland, USA (1996)

    Google Scholar 

  46. Business Intelligence Cup - 2004: Organized by the University of Chile, http://www.tis.cl/bicup_04/text-bicup/BICUP/202004/20public/20data.zip

  47. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvm

  48. Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: YALE: Rapid Prototyping for Complex Data Mining Tasks. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2006 (2006)

    Google Scholar 

  49. Fawcett, T.: An introduction to ROC analysis. Pattern Recognition Letters 27, 861–874 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Development and Research in banking Technology, Castle Hills Road #1, Masab Tank, Hyderabad, 500 057, AP, India

    M. A. H. Farquad & V. Ravi

  2. Department of Computer & Information Sciences, University of Hyderabad, Hyderabad, 500 046, AP, India

    M. A. H. Farquad & S. Bapi Raju

Authors
  1. M. A. H. Farquad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. Ravi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Bapi Raju
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Engineering, The Parade, Cardiff University, CF24 3AA, Cardiff, UK

    Rossitza Setchi

  2. Dept. of Computer Science and Software Engineering, BUckingham Building, Lion Terrace, University of Portsmouth, PO1 3HE, Portsmouth, UK

    Ivan Jordanov

  3. KES International, 145-157, St. John Street, EC1V 4PY, London, UK

    Robert J. Howlett

  4. School of Electrical and Information Engineering, University of South Australia, ,, Adelaide, Mawson Lakes Campus, 5095, SA, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Farquad, M.A.H., Ravi, V., Raju, S.B. (2010). Rule Extraction from Support Vector Machine Using Modified Active Learning Based Approach: An Application to CRM. In: Setchi, R., Jordanov, I., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based and Intelligent Information and Engineering Systems. KES 2010. Lecture Notes in Computer Science(), vol 6276. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15387-7_50

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-15387-7_50

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-15386-0

  • Online ISBN: 978-3-642-15387-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation