Automatic MLP Weight Regularization on Mineralization Prediction Tasks

Abstract

Conventional neural network training methods attempt to find a set of values for the network weights by minimizing an error function using some gradient descent based technique. In order to achieve good generalization performance, it is usually necessary to introduce a regularization term into the error function to prevent weights becoming overly large. In the conventional approach, the regularization coefficient, which controls the degree to which large weights are penalized, must be optimized outside of the weight training procedure, and this is usually done by means of a cross-validation procedure in which some training examples are held out, thereby reducing the number of examples available for weight optimization. Bayesian methods provide a means of optimizing these coefficients within the weight optimization procedure. This paper reports on the application of Bayesian MLP techniques to the task of predicting mineralization potential from geoscientific data. Results demonstrate that the Bayesian approach results in similar maps to the conventional MLP approach, while avoiding the complex cross-validation procedure required by the latter.