Convolutional Neural Network Architecture with Exact Solution

Abstract

Convolutional Neural Networks (CNNs) have been explored rigorously, due to their complex image classification capabilities and applied in many real-world applications. In majority of such applications, training of CNN using a back-propagation type iterative learning has become a standard practice, but this makes training of CNN very inefficient and uncertain because of various problems such as local minima and paralysis. Other iterative and non-iterative learning including exact solution-based learning might be more efficient in terms of accuracy and certainty in training, however, potential of this type of combined learning has not been fully explored by CNN researchers. Therefore, in this paper an exact solution based new convolutional neural network architecture is proposed. In proposed architecture, a novel concept is introduced in which the weights of CNN layers are updated using iterative process for a fixed number of epochs and then the weights of fully connected layer are calculated using an exact solution process. Both iterative and calculated weights are then used for training the full architecture. The proposed approach has been evaluated on three benchmark datasets such as CIFAR-10, MNIST and Digit. The experimental results have demonstrated that the proposed approach can achieve higher accuracy than the standard CNN. Statistical significance test was carried out to prove the efficacy of proposed approach.