An Electronic Commerce Application of the Bayesian Framework for MLPs: The Effect of Marginalisation and ARD

Within the Bayesian approach to the training of multi-layer perceptrons for classification problems, the interpretation of the outputs as posterior probabilities of class-membership requires us to integrate out (marginalise) the network function over the distribution of network weights. MacKay [1] suggests an approximation of such an analytically intractable integral, in which the integration is over the network output preactivations. The network predictions can be over-optimistic if this process of marginalisation is ignored. This study attempts to assess the effect of marginalisation, with the approximation mentioned above, on two Bayesian neural network models: one with a single regularisation term; and another giving way to a process known as

Automatic Relevance Determination (ARD), with multiple regularisation terms. A real-world classification problem, concerning the discrimination of online purchasers and non-purchasers using Internet’s WWW users’ opinions, is the test-bed for this assessment.