Skip to main content
SpringerLink
Log in

Using artificial neural networks to enhance CART

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Accuracy is a critical factor in predictive modeling. A predictive model such as a decision tree must be accurate to draw conclusions about the system being modeled. This research aims at analyzing and improving the performance of classification and regression trees (CART), a decision tree algorithm, by evaluating and deriving a new methodology based on the performance of real-world data sets that were studied. This paper introduces a new approach to tree induction to improve the efficiency of the CART algorithm by combining the existing functionality of CART with the addition of artificial neural networks (ANNs). Trained ANNs are utilized by the tree induction algorithm by generating new, synthetic data, which have been shown to improve the overall accuracy of the decision tree model when actual training samples are limited. In this paper, traditional decision trees developed by the standard CART methodology are compared with the enhanced decision trees that utilize the ANN’s synthetic data generation, or CART+. This research demonstrates the improved accuracies that can be obtained with CART+, which can ultimately improve the knowledge that can be extracted by researchers about a system being modeled.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Chih-Hung H, Mao-Jiun JW (2005) Using decision tree-based data mining to establish a sizing system for the manufacture of garments. Int J Adv Manuf Technol 26(5):669–674

    Article  Google Scholar 

  2. Witten IH, Frank E (2011) Data mining: practical machine learning tools and techniques 3rd edn. Morgan Kaufmann Series in Data Management Systems, Morgan Kaufmann, Burlington, MA

  3. Kriegel H-P, Borgwardt KM, Kröger P, Pryakhin A, Schubert M, Zimek A (2007) Future trends in data mining. Data Min Knowl Discov 15(1):87–97

    Article  MathSciNet  Google Scholar 

  4. Lavrac N (2004) Introduction: lesson learned from data mining applications and collaborative problem solving. Mach Learn 57:13–34

    Article  MATH  Google Scholar 

  5. Kusiak A, Smith M (2007) Data mining in design of products and production systems. Ann Rev Control 31:147–156

    Article  Google Scholar 

  6. Brand E, Gerritsen R (1998) Decision trees, DBMS, data mining solutions supplément, http://www.dbmsmag.com/9807m05.html

  7. Apte C, Weiss S (1997) Data mining with decision trees and decision rules. Futur Gener Comput Syst 13:197–210

    Article  Google Scholar 

  8. Li Y (2006) Predicting materials properties and behavior using classification and regression trees. Mater Sci Eng A 433:261–268

    Article  Google Scholar 

  9. Classification and Regression Trees (C&RT) available at http://www.statsoft.com/textbook/stcart.html

  10. Wilkinson L (1992) Tree structured data analysis: AID, CHAID and CART, Department of Statistics, Northwestern University, Evanston, IL 60201

  11. Shih Y-S (2005) QUEST classification tree available at http://www.stat.wisc.edu/~loh/quest.html

  12. Feldman D, Gross S (2005) Mortgage default: classification trees analysis. J Real Estate Fin Econ 30(4):369–396

    Article  Google Scholar 

  13. Questier F, Put R, Coomans D, Walczak B, Vander Heydan Y (2005) The user of CART and multivariate regression trees for supervised and unsupervised feature selection. Chemom Intell Lab Syst 76:45–54

    Article  Google Scholar 

  14. Caetanoa S, Aires-de-Sousab J, Daszykowskia M, Vander Heydena Y (2005) Prediction of enantioselectivity using chirality codes and classification and regression trees. Anal Chim Acta 544:315–326

    Article  Google Scholar 

  15. Andryashin A (2005) Financial applications of classification and regression trees, MS, Thesis, Center of Applied Statistics and Economics, Humboldt University, Berlin

  16. Waheed T, Bonnell RB, Prasher SO, Paulet E (2006) Measuring performance in precision agriculture: CART—a decision tree approach. Agric Water Manage 84:173–185

    Article  Google Scholar 

  17. Provost F, Danyluk A (1999) Problem definition, data cleaning, and evaluation: a classifier learning case study. Informatica 23:123–136

    Google Scholar 

  18. Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and regression trees. Wadsworth, Inc., Monterey, CA

  19. Hsu C-H, Wang M-JJ (2005) Using decision tree-based data mining to establish a sizing system for the manufacture of garments. Int J Adv Manuf Technol 26:669–674

    Article  Google Scholar 

  20. Evans B, Fisher D (1994) Overcoming process delays with decision tree induction. IEEE Expert 9:60–66

    Article  Google Scholar 

  21. Wu M-C, Lin S-Y, Lin C-H (2006) An effective application of decision tree to stock trading. Expert Syst Appl 31:270–274

    Article  Google Scholar 

  22. Yang B-S, Lin DS, Tan ACC (2005) VIBEX: an expert system for vibration fault diagnosis of rotating machinery using decision tree and decision table. Expert Syst Appl 28:735–742

    Article  Google Scholar 

  23. McSherry D (1999) Strategic induction of decision trees. Knowl Based Syst 12:269–275

    Article  Google Scholar 

  24. Changnon D, Ritsche M, Elyea K, Shelton S, Schramm K (2000) Integrating climate forecasts and natural gas supply information Into a natural gas purchasing decision. Meterol Appl 7:211–216

    Article  Google Scholar 

  25. Moon TH, Sohn SY (2005) Intelligent approach for effective management of governmental funds for small and medium enterprises. Expert Syst Appl 29(566):572

    Google Scholar 

  26. Hsia T-C, Shie A-J, Chen L-C (2008) Course planning of extension education to meet market demand by using data mining techniques—an example of Chinkuo technology university in Taiwan. Expert Syst Appl 34:596–602

    Article  Google Scholar 

  27. Rangwala MH (2006) Empirical investigation of decision tree extraction from neural networks, MS, Thesis, Department of Manufacturing and Systems Engineering, Ohio University, Athens, OH

  28. Michie D (1989) Problems of computer-aided concept formation. In: Quinlan JR (ed) Applications of expert systems volume 2. Addison Wesley, Wokingham, UK, pp 310–333

  29. Gardner MW, Dorling SR (1998) Artificial neural networks (the multilayer Perceptron)—a review of applications in the atmospheric sciences. Atmos Environ 32:2627–2636

    Article  Google Scholar 

  30. Aakerlund L, Hemmingsen R (1998) Neural networks as models of psychopathology. Biol Psychiatr 43(7):471–482

    Article  Google Scholar 

  31. Yang CC, Prasher SO, Landry JA, Ramaswamy HS, Tommasi D (2000) Application of artificial neural networks in image recognition and classification of crop and weeds. Can Agric 13 Eng 42:147–152

    Google Scholar 

  32. Laskaria EC, Meletiou GC, Tasoulis DK, Vrahatis MN (2006) Studying the performance of artificial neural networks on problems related to cryptography. Nonlinear Anal Real World Appl 7:937–942

    Article  MathSciNet  Google Scholar 

  33. Madani K, Braz J et al (eds) Industrial and real world applications of artificial neural networks, illusion or reality? Informatics in control. Autom Robot I:11–26

  34. Allex CF, Shavlik JW, Blattner FR (1999) Neural network input representations that produce accurate consensus sequences from DNA fragment assemblies. Bioinformatics 15(9):723–728

    Article  Google Scholar 

  35. An Introduction to Neural Networks, Prof Leslie Smith, Centre for Cognitive and Computational Neuroscience available at http://www.cs.stir.ac.uk/~lss/NNIntro/InvSlides.html

  36. Krawiec K, Slowinski R, Szczesniak I (1998) Pedagogical method for extraction of symbolic knowledge from neural networks. In: Polkowski L, Skowron A (eds) Proceedings of the first international conference on rough sets and current trends in computing (RSCTC '98). Springer, London, UK

  37. Bhagat PM (2005) Pattern recognition in industry. Elsevier. ISBN 0-08-044538-1

  38. Craven MW (1996) Extracting comprehensible models from trained neural networks, PhD Thesis, Computer Science Department, University of Wisconsin, Madison, WI

  39. Dwyer K, Holte R Decision tree instability and active learning. Department of Computing Science, University of Alberta, Edmonton AB, Canada

  40. Neural Networks by Christos Stergiou and Dimitrios Siganos at http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html

  41. Artificial Neural Networks available at www.makhfi.com/tutorial/introduction.htm

  42. Nicholls JG, Martin AR, Wallace BG, Fuchs PA (2001) From neuron to brain, 4th ed. Sinauer Associates, Sunderland, MA. ISBN 0878934391

  43. Artificial Neural Networks at https://www.dacs.dtic.mil/techs/neural/neural2.php

  44. Chen F (2004) Learning accurate and understandable rules from SVM classifiers, MS, Thesis, School of Computer Science, Simon Fraser University

  45. Luin AD (1988) Queries and concept learning. Mach Learn 2:319–342

    Google Scholar 

  46. Murphy PM (1995) UCI repository of machine learning databases—a machinereadable data repository, maintained at the Department of Information and Computer Science, University of California, Irvine., anonymous FTP from ics.uci.edu in the directory pub/machinelearningdatabases

  47. Heinz G, Peterson LJ, Johnson RW, Kerk CJ (2003) Exploring relationships in body dimensions. J Stat Educ 11(2). American Statistical Association Website at http://www.amstat.org/publications/jse

  48. Millie DF, Weckman GR, Pigg RJ, Teste PA, Dyble J, Litaker RW, Carrick HJ, Fahnestiel GL (2006) Modeling phytoplankton abundance in Saginaw bay, Lake Huron: using artificial neural networks to discern functional influence of environmental variables and relevance to a great lakes observing system. J Phycol 42:336–349

    Article  Google Scholar 

  49. Hernandez S, Nesic S, Weckman G, Ghai V (2005) Use of artificial neural networks for predicting crude oil effect on CO2 corrosion of carbon steels. Corrosion. Submitted and accepted for publication, April 2006

  50. Rastogi P (2005) Assessing wireless network dependability using neural networks, MS, Thesis., School of Communication, Ohio University, Athens, OH

  51. Weckman G, Snow A, Rastogi P, Rangwala M (2008) Assessing wireless network dependability through knowledge extraction via decision trees. IARIA: ICONS 2008

  52. CART Pro 6.0 available at http://www.salford-systems.com

  53. Classification and Regression trees available at http://www.statsoft.com/textbook/stathome.html

  54. Maindonald J, Braun J (2003) Data analysis and graphics using R: an example-based approach. Cambridge University Press, Cambridge, UK

  55. NeuroDimensions, NeuroSolutions v5.07, available at http://www.nd.com

  56. Rangwala M (2006) Empirical investigation of decision tree extraction from neural networks, MS, Thesis, Russ College of Engineering and Technology, Department of Industrial and Systems Enginnering, Ohio Univerity, Athens, OH

Download references

Author information

Authors and Affiliations

  1. Department of Management Systems, College of Business, Ohio University, Copeland Hall, Athens, OH, 45701-2979, USA

    William A. Young II

  2. Department of Industrial and Systems Engineering, Russ College of Engineering and Technology, Ohio University, Stocker Center, Athens, OH, 45701-2979, USA

    Gary R. Weckman, Vijaya Hari & Harry S. Whiting II

  3. The J. W. McClure School of Information and Telecommunication Systems, Ohio University, Lindley Hall, Athens, OH, 45701-2979, USA

    Andrew P. Snow

Authors
  1. William A. Young II
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gary R. Weckman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vijaya Hari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Harry S. Whiting II
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andrew P. Snow
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William A. Young II.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Young, W.A., Weckman, G.R., Hari, V. et al. Using artificial neural networks to enhance CART. Neural Comput & Applic 21, 1477–1489 (2012). https://doi.org/10.1007/s00521-012-0887-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-0887-4

Keywords

Search

Navigation