Convergence of Mini-Batch Learning for Fault Aware RBF Networks

Abstract

In between online and batch modes, there is a mini-batch concept that takes a subset of the training samples for updating the weights at each iteration. Traditional analysis of mini-batch is based on the stochastic gradient descent in which we assume that the process of taking mini-batches is performed in a random manner. In fact, practically, the mini-batch process is not in a random manner. In the last decade, many online and batch learning algorithms for fault aware radial basis function (RBF) networks. However, not much works on mini-batch for fault aware RBF networks are reported. This paper proposes a mini-batch learning algorithm for fault aware RBF networks. In our approach, rather than using the assumptions of the stochastic gradient descent, we consider that the partitions of mini-batches are fixed, and that those mini-batches are presented in a fixed order. Even with the above fixed arrangement, we are still able to prove that the training weight vector converges to the fault aware batch mode solution. In addition, we present the sufficient condition for the convergence.