Skip to main content
SpringerLink
Log in

Using a heuristic approach to derive a grey-box model through an artificial neural network knowledge extraction technique

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Artificial neural networks (ANNs) are primarily used in academia for their ability to model complex nonlinear systems. Though ANNs have been used to solve practical problems in industry, they are not typically used in nonacademic environments because they are not very well understood, complicated to implement, or have the reputation of being a “black-box” model. Few mathematical models exist that outperform ANNs. If a highly accurate model can be constructed, the knowledge should be used to understand and explain relationships in a system. Output surfaces can be analyzed in order to gain additional knowledge about a system being modeled. This paper presents a systematic approach to derive a “grey-box” model from the knowledge obtained from the ANN. A database for an automobile’s gas mileage performance is used as a case study for the proposed methodology. The results show a greater ability to generalize system behavior than other benchmarked methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Young W, Weckman G, Thompson J, Brown M (2008) Artificial neural networks for knowledge extraction of concrete shear strength prediction. Int J Ind Eng 15(1):1–10

    MATH  Google Scholar 

  2. Hernandez S, Nesic S, Weckman G, Ghai V (2006) Use of artificial neural networks for predicting crude oil effect on CO2 corrosion of carbon steels. Corros J 62(6):1–35

    Google Scholar 

  3. Snow A, Weckman G, Chayanam K (2006) Modeling telecommunication outages due to power loss. Int J Ind Eng Theory Appl Pract 13(1):51–60

    Google Scholar 

  4. Weckman G, Lakshminarayanan S, Snow A, Marvel J (2008) An integrated stock market forecasting model using neural networks. Int J Business Forecast Mark Intell 1(1). doi:10.1504/IJBFMI.2008.020813

  5. Millie D, Weckman G, Paerl H, Pinckney J, Bendis B, Pigg R et al (2006) Neural network modeling of estuarine indicators: hindcasting phytoplankton biomass and net ecosystem production in the Neuse (North Carolina) and Trout (Florida) Rivers. Ecol Indic 6:589–608. doi:10.1016/j.ecolind.2005.08.021

    Article  Google Scholar 

  6. Ozesmi S, Ozesmi U (1999) An artificial neural network approach to spatial habitat modeling with interspecific interaction. Ecol Model 116:15–31. doi:10.1016/S0304-3800(98)00149-5

    Article  Google Scholar 

  7. Box G, Draper N (1987) Empirical model-building and response surfaces. Wiley, New York

    MATH  Google Scholar 

  8. National Aeronautics Space Administration (2004) NASA cost estimation handbook. National Aeronautics and Space Administration, Washington, DC

    Google Scholar 

  9. Huysmans J, Baesens B, Vanthienen J (2006) Using rule extraction to improve the comprehensibility of predictive models. FETEW Internal Research. K.U.Leuven, Leuven

    Google Scholar 

  10. Xiong Q, Jutan A (2002) Grey-box modelling and control of chemical processes. Chem Eng Sci 57:1027–1039. doi:10.1016/S0009-2509(01)00439-0

    Article  Google Scholar 

  11. Oussar Y, Dreyfus G (2001) How to be a gray box: dynamic semi-physical modeling. Neural Netw 14:1161–1172. doi:10.1016/S0893-6080(01)00096-X

    Article  Google Scholar 

  12. Francisco C, Acuña G, Cubillos F (2007) Indirect training of grey-box models: application to a bioprocess. In: Cruz F, Acuña G, Cubillos F, Moreno V, Bassi D (eds) Lecture Notes in Computer Science. Springer, Berlin/Heidelberg, pp 391–397

    Google Scholar 

  13. Acuña G, Cubillos F, Thibault J, Latrille E (1999) Comparison of methods for training grey-box neural network models. Comput Chem Eng Suppl 23:561–564

    Article  Google Scholar 

  14. Johannet A, Vayssade B, Bertin D (2007) Neural networks: from black box toward transparent box application to evapotranspiration modeling. Proc World Acad Sci Eng Technol 24:162–169

    Google Scholar 

  15. Tornøe C, Jacobsen J, Pedersen O, Hansen T, Madsen H (2004) Grey-box modelling of pharmacokinetic/pharmacodynamic systems. J Pharmacokinet Pharmacodyn 31(5):401–417. doi:10.1007/s10928-004-8323-8

    Article  Google Scholar 

  16. Geeraerd AH, Herremans CH, Cenens C, Van Impe JF (1998) Application of artificial neural networks as a non-linear modular modeling technique to describe bacterial growth in chilled food products. Int J Food Microbiol 44:49–68. doi:10.1016/S0168-1605(98)00127-5

    Article  Google Scholar 

  17. Reed RD, Marks RJ (1998) Neural smithing: supervised learning in feedforward artificial neural networks. MIT Press, Cambridge

    Google Scholar 

  18. Hornik H, Stinchcomb M, White J (1982) Multilayer feedforward networks are universal approximators. Neural Netw 2:359–366. doi:10.1016/0893-6080(89)90020-8

    Article  Google Scholar 

  19. Meireles M, Almeida P, Simões M (2003) A comprehensive review for industrial applicability of artificial neural networks. IEEE Trans Ind Electron 50(3):585–601. doi:10.1109/TIE.2003.812470

    Article  Google Scholar 

  20. Lek S, Guegan J (1999) Artificial neural networks as a tool in ecological modeling, an introduction. Ecol Modell 120:65–73. doi:10.1016/S0304-3800(99)00092-7

    Article  Google Scholar 

  21. Lange N (1999) New mathematical approaches in hydrological modeling—an application of artificial neural networks. Phys Chem Earth Part B Hydrol Oceans Atmos 24(1):31–35. doi:10.1016/S1464-1909(98)00007-0

    Article  Google Scholar 

  22. Browne A, Hudson B, Whitley D, Ford M, Picton P, Kazemian H (2003) Knowledge extraction from neural networks. In: Proceedings of the 29th Annual Conference of the IEEE Industrial Electronics Society. Roanoke, Virginia, pp. 1909–1913

  23. Garson D (1991) Interpreting neural-network connection weights. AI Expert 6(4):47–51

    Google Scholar 

  24. Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Syst Man Cybern 30(4):451–462. doi:10.1109/5326.897072

    Article  Google Scholar 

  25. Cristea A, Cristea P, Okamoto T (1997) Neural networks knowledge extraction. Electrotech Eng 42(2):477–491

    Google Scholar 

  26. Towell G, Shavlik J (1993) The extraction of re¯ned rules from knowledge-based neural networks. Mach Learn 13(1):71–101

    Google Scholar 

  27. Setiono R, Liu H (1997) Neurolinear: from neural networks to oblique decision rules. Neural Comput 17(1):1–24

    Google Scholar 

  28. Setiono R, Leow W (2000) FERNN: an algorithm for fast extraction of rules from neural networks. Appl Intell 12(1–2):15–25. doi:10.1023/A:1008307919726

    Article  Google Scholar 

  29. Saito K, Nakano R (2002) Extracting regression rules from neural networks. Neural Netw 15(10):1279–1288. doi:10.1016/S0893-6080(02)00089-8

    Article  Google Scholar 

  30. Breiman L, Friedman H, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth, Beltnont

    MATH  Google Scholar 

  31. Quinlan J (1988) C4.5: programs for machine learning. Morgan Kaufmann, USA

  32. Craven M (1996) Extracting comprehensible models from trained neural networks. Ph.D. Dissertation, University of Wisconsin, Madison, Wisconsin

  33. Schmitz G, Aldrich C, Gouws F (1999) ANN-DT: an algorithm for extraction of decision trees from artificial neural networks. IEEE Trans Neural Netw 10(6):1392–1401. doi:10.1109/72.809084

    Article  Google Scholar 

  34. Timofeev R (2004) Classification and Regression Trees (CART) Theory and Applications. Masters Thesis, Humboldt University, Center of Applied Statistics and Economics, Berlin

  35. Quinlan J (1986) Induction of decision trees. In: Mitchell T (ed) Mach Learn, vol 1. McGraw-Hill, New York, pp 81–106

    Google Scholar 

  36. Ziarko W, Yao Y (2001) A comparison of several approaches to missing values in data mining in Rough Sets and current trends in computing. Springer, Berlin

    Google Scholar 

  37. Rangwala M (2006) Empirical investigation of decision tree extraction from neural networks. Ohio University, Department of Industrial and Systems Engineering, Athens, OH, USA

  38. Zhou ZH, Jiang Y, Chen SF (2003) Extracting symbolic rules from trained neural network ensembles. AI Commun 16(1):3–15

    Google Scholar 

  39. Nath R, Rajagopalan B, Ryker R (1997) Determining the saliency of input variables in neural network classifiers. Comput Oper Res 24(8):767–773. doi:10.1016/S0305-0548(96)00088-3

    Article  MATH  Google Scholar 

  40. Thrun S (1995) Extracting rules from artificial neural networks with distributed representations. In: Tesauro G, Touretzky D, Leen T (eds) Advances in Neural Information Processing Systems 7. MIT Press, Cambridge

    Google Scholar 

  41. Lasdon L, Waren D, Jain A, Ratner M (1978) Design and testing of a generalized reduced gradient code for nonlinear programing. ACM Trans Math Softw 4(1):34–49. doi:10.1145/355769.355773

    Article  MATH  Google Scholar 

  42. Watson J, Fylstra D (1996) Solver user’s guide. Frontline Systems, Incline Village

    Google Scholar 

  43. Windward Technolgies and Optimal Methods (1997) User’s guide for GRG2 optimization library. Retrieved April 7, 2007, from http://www.maxthis.com/Grg2ug.htm

  44. Fylstra D, Lasdon L, Watson J (1998) Design and use of the microsoft excel solver. Interfaces 28(5):29–55. doi:10.1287/inte.28.5.29

    Article  Google Scholar 

  45. Lasdon L, Smith S (1992) Solving largesparse nonlinear programs using GRG. ORSA J Comput 1:2–15

    MathSciNet  Google Scholar 

  46. Newman D, Hettich S, Blake C, Merz C (1998) UCI repository of machine learning databases. (I. University of California, Producer, and Department of Information and Computer Sciences) Retrieved April 5, 2007, from ftp://ftp.ics.uci.edu/pub/machine-learning-databases/auto-mpg/

  47. NeuroDimension (2006) Intelligent Software Solutions. Retrieved 2007, from http://www.nd.com/

  48. Gill P, Murray W, Wright M (1981) Practical optimization. Academic Press, London

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial and Systems Engineering, Russ College of Engineering and Technology, Stocker Center, Ohio University, Athens, OH, 45701-2979, USA

    William A. Young II & Gary R. Weckman

Authors
  1. William A. Young II
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gary R. Weckman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gary R. Weckman.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Young, W.A., Weckman, G.R. Using a heuristic approach to derive a grey-box model through an artificial neural network knowledge extraction technique. Neural Comput & Applic 19, 353–366 (2010). https://doi.org/10.1007/s00521-009-0270-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-009-0270-2

Keywords

Search

Navigation