Abstract
In domains with limited data, such as ballistic impact, prior researches have proven that the optimization of artificial neural models is an efficient tool for improving the performance of a classifier based on MultiLayer Perceptron. In addition, this research aims to explore, in the ballistic domain, the optimization of other machine learning strategies and their application in regression problems. Therefore, this paper presents an optimization methodology to use with several approaches of machine learning in regression problems, maximizing the limited dataset and locating the best network topology and input vector of each network model. This methodology is tested in real regression scenarios of ballistic impact with different artificial neural models, obtaining substantial improvement in all the experiments. Furthermore, the quality stage, based on criteria of information theory, enables the determination of when the complexity of the network design does not penalize the fit over the data and thereby the selection of the best neural network model from a series of candidates. Finally, the results obtained show the relevance of this methodology and its application improves the performance and efficiency of multiple machine learning strategies in regression scenarios.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
ABAQUS (2003) ABAQUS/Explicit v6.4 users manual. ABAQUS Inc, Richmond, USA
Ahn B, Cho S, Kim C (2000) The integrated methodology of rough set theory and artificial neural network for business failure prediction. Expert Syst Appl 18(2):65–74
Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716–723
An G (1996) The effects of adding noise during backpropagation training on a generalization performance. Neural Comput 8(3):643–674
Anlauf J, Biehl M (1989) The adatron: an adaptive perceptron algorithm. Europhys Lett 10:687–692
Bermejo P, Joho H, Joemon M, Villa R (2008) Comparison of feature construction methods for video relevance prediction. Adv Multimed Model, pp 185–196
Bisagni C, Lanzi L, Ricci S (2002) Optimization of helicopter subfloor components under crashworthiness requirements using neural networks. J Aircr 39(2):296–304
Bishop C (1995) Training with noise is equivalent to Tikhonov regularization. Neural Comput 7:108
Bishop C (1996) Neural networks for pattern recognition. Oxford University Press, USA
Boger Z, Weber R (2000) Finding an optimal artificial neural network topology in real-life modeling—two approaches. In: ICSC symposium on neural computation
Breiman L, Spector P (1992) Submodel selection and evaluation in regression the x-random case. Int Stat Rev 60:291–319
Burger M, Neubauer A (2003) Analysis of Tikhonov regularization for function approximation by neural networks. Neural Netw 16(1):79–90
Carcenac M (2009) A modular neural network for super-resolution of human faces. Appl Intell 30:168–186. doi:10.1007/s10489-007-0109-4
Cho SB, Shimohara K (1998) Evolutionary learning of modular neural networks with genetic programming. Appl Intell 9:191–200
Cirrincione G, Cirrincione M (2003) A novel self-organizing neural network for motion segmentation. Appl Intell 18:27–35. doi:10.1023/A:1020970617241
Cover T (1965) Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans Electron Comput EC-14(3):326–334
Dietterich T (1998) Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput 10:1895–1923
Diosan L, Rogozan A, Pecuchet JP Improving classification performance of support vector machine by genetically optimising kernel shape and hyper-parameters. App Intell (2010). doi:10.1007/s10489-010-0260-1
Durand N, Alliot JM, Médioni F (2000) Neural nets trained by genetic algorithms for collision avoidance. Appl Intell 13:205–213
Feng C, Yu Z, Kingi U, Baig M (2005) Threefold vs fivefold cross validation in one-hidden-layer and two-hidden-layer predictive neural network modeling of machining surface roughness data. J Manuf Syst 24(2):93–107
Ferri C, Hernández-Orallo J, Modroiu R (2009) An experimental comparison of performance measures for classification. Pattern Recognit Lett 30(1):27–38
Foley D (1972) Considerations of sample and feature size. IEEE Trans Inf Theory 18(5):618–626
Garcia-Crespo A, Ruiz-Mezcua B, Fernandez D, Zaera R (2006) Prediction of the response under impact of steel armours using a multilayer perceptron. Neural Comput Appl 16(2):147–154
Garcia-Crespo A, Ruiz-Mezcua B, Gonzalez-Carrasco I, Lopez-Cuadrado J (2009) Multilayer perceptron training optimization for high speed impacts classification. Springer, Netherlands, pp 377–388
Gevrey M, Dimopoulos I, Lek S (2003) Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecol Model 160(16):249–264
Girosi F, Jones M, Poggio T (1995) Regularization theory and neural networks architectures. Neural Comput 7:219–269
Gomez-Ruiz J, Jerez-Aragones J, Muñoz-Perez J, Alba-Conejo E (2004) A neural network based model for prognosis of early breast cancer. Appl Intell 20(3):231–238
Gonzalez-Carrasco I (2010) Análisis, optimización y evaluación de modelos de redes de neuronas artificiales para la clasificación y predicción de impactos de alta velocidad sobre distintos materiales. PhD thesis, Universidad Carlos III de Madrid
Gonzalez-Carrasco I, Garcia-Crespo A, Ruiz-Mezcua B, Lopez-Cuadrado J (2009) Dealing with limited data in ballistic impact scenarios: an empirical comparison of different neural network approaches. Appl Intell, 1–21
Grochowski M, Dutch W (2007) Learning highly non-separable boolean functions using constructive feedforward neural network. In: Artificial neural networks. ICANN 2007. Springer, Berlin. pp 180–189
He Y, Sun Y (2001) Neural network-based l1-norm optimisation approach for fault diagnosis of nonlinear circuits with tolerance. IEE Proc, Circuits Devices Syst 148(4):223–228
Henderson C, Potter W, McClendon R, Hoogenboom G (2000) Predicting aflatoxin contamination in peanuts: a genetic algorithm/neural network approach. Appl Intell 12:183–192
Holldobler S, Kalinke Y, Storr H (1999) Approximating the semantics of logic programs by recurrent neural networks. Appl Intell 11:45–58. doi:10.1023/A:1008376514077
Holmstrom L, Koistinen P (1992) Using additive noise in back-propagation training. IEEE Trans Neural Netw 3(1):24–38
Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366
Ince R (2004) Prediction of fracture parameters of concrete by artificial neural networks. Eng Fract Mech 71(15):2143–2159
Johnson G, Cook W (1983) A constitutive model and data for metals subjected to large strains, high strain rates, and temperatures. In: International symposium ballistics
Jordan M (1986) Serial order: a parallel distributed processing approach. Tech rep, University of California, San Diego
Kaikhah K, Garlick R (2000) Variable hidden layer sizing in Elman recurrent neuro-evolution. Appl Intell 12:193–205. doi:10.1023/A:1008315023738
Karystinos G, Pados D (2000) On overfitting generalization, and randomly expanded training sets. IEEE Trans Neural Netw 11(5):1050–1057
Kohonen T (1988) An introduction to neural computing. Neural Netw 1(1):3–16
Koistinen P, Holmstrom L (1991) Kernel regression and backpropagation training with noise. In: IEEE international joint conference on neural networks, vol 1, pp 367–372
Krabbes M, Döschner C (2002) Modelling of complete robot dynamics based on a multi-dimensional, rbf-like neural architecture. Appl Intell 17:61–73. doi:10.1023/A:1015779731969
Lanzi L, Bisagni C, Ricci S (2004) Neural network systems to reproduce crash behavior of structural components. Comput Struct 82(1):93–108
Lefebvre C, Fancourt C, Principe JJG (2007) Neuro solutions documentation
Liu H, Setiono R (1998) Incremental feature selection. Appl Intell 9(3):217–230
Liu S, Huang J, Sung J, Lee C (2002) Detection of cracks using neural networks and computational mechanics. Comput Methods Appl Mech Eng 191(25–26):2831
Majumder M, Roy P, Mazumdar A (2007) Optimization of the water use in the river Damodar in West Bengal in India: an integrated multi-reservoir system with the help of artificial neural network. Eng Comput Archit 1(2)
Mandal S, Saha D, Banerjee T (2005) A neural network based prediction model for flood in a disaster management system with sensor networks. In: Proceedings of 2005 international conference on intelligent sensing and information processing, pp 78–82
Mandic D, Chambers J (2001) Recurrent neural networks for prediction: learning algorithms, architectures and stability. Wiley, New York
Maniezzo V (1994) Genetic evolution of the topology and weight distribution of neural networks. IEEE Trans Neural Netw 5(1):39–53
Matsuoka K (1992) Noise injection into inputs in back-propagation learning. IEEE Trans Syst Man Cybern 22(3):436–440
Mrazova I, Wang D (2007) Improved generalization of neural classifiers with enforced internal representation. Neurocomputing 70(16-18):2940–2952
Ośullivan D, Dooley L (2010) Collaborative innovation for the management of information technology resources. Int J Hum Cap Inf Technol Prof 1(1):16–30
Park J, Sandberg I (1991) Universal approximation using radial-basis-function networks. Neural Comput 3(2):246–257
Priddy K, Keller P (2005) Artificial neural networks: an introduction. SPIE Press, Bellingham
Principe J, Euliano N, Lefebvre W (1999) Neural and adaptive systems: fundamentals through simulations with CD-ROM. Wiley, New York
Pujol J, Poli R (1998) Evolving the topology and the weights of neural networks using a dual representation. Appl Intell 8:73–84
Rissanen J (1978) Modeling by shortest data description. Automatica 14:445–471
Roy N, Potter W, Landau D (2004) Designing polymer blends using neural networks genetic algorithms, and Markov chains. Appl Intell 20:215–229
Sokolova M, Rasras R, Skopin D (2006) The artificial neural network based approach for mortality structure analysis. Am J Appl Sci 3(2):1698–1702
Son J, Lee D, Kim I, Choi S (2004) A study on genetic algorithm to select architecture of a optimal neural network in the hot rolling process. J Mater Process Technol 153–154, 643–648. Proceedings of the international conference in advances in materials and processing technologies
Songwu L, Member S, Basar T (1998) Robust nonlinear system identification using neural network models. IEEE Trans Neural Netw 9:407
Swingler K (1996) Applying neural networks. A practical guide. Academic Press, San Diego
Tang B, Hsieh W, Monahan A, Tangang F (2000) Skill comparisons between neural networks and canonical correlation analysis in predicting the equatorial pacific sea surface temperatures. J Climate 13:287–293
Tarassenko L (1998) A guide to neural computing applications. Arnol/NCAF
Ueda N (2000) Optimal linear combination of neural networks for improving classification performance. IEEE Trans Pattern Anal Mach Intell 22(2):207–215
Vapnik V (1995) The nature of statistical learning theory. Springer, Berlin
Waszczyszyn Z, Ziemianski L (2001) Neural networks in mechanics of structures and materials—new results and prospects of applications. Comput Struct 79(16):2261–2276
West D (2000) Neural network credit scoring models. Comput Oper Res 27(11–12):1131–1152
Xie C-L, Chang J-Y, Shi, X-C, Dai, J-M (2010) Fault diagnosis of nuclear power plant based on genetic-rbf neural network. Int J Comput Appl Technol 39(1–3), 159–165
Zhang L, Subbarayan G (2002) An evaluation of back-propagation neural networks for the optimal design of structural systems: part i. Training procedures. Comput Methods Appl Mech Eng 191(14), 2873–2886
Zukas J (1990) High velocity impact dynamics. Wiley, New York
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gonzalez-Carrasco, I., Garcia-Crespo, A., Ruiz-Mezcua, B. et al. An optimization methodology for machine learning strategies and regression problems in ballistic impact scenarios. Appl Intell 36, 424–441 (2012). https://doi.org/10.1007/s10489-010-0269-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-010-0269-5