Image classification with a MSF dropout

Abstract

In recent years, as the main carrier of deep learning, Deep Neural Network has attracted the attention of experts in computer field. The application of deep neural network can effectively solve complex problems in life. However, in the process of training, the complex relationship caused by noisy data leads to an overfitting, which can impact the robustness of network model. Dropout, as one kind of random regularization techniques, carries a significant effect on restraining the overfitting of deep neural network. The traditional standard dropout can restrain the overfitting in a simple and quick way, but the accuracy is impacted because it cannot accurately locate the appropriate scale. This paper proposes a multi-scale fusion (MSF) dropout method on the basis of standard dropout. At first, several groups of network model with different combinations of dropout rates were trained; then the improved genetic algorithm was used to calculate the optimal scale of each network model; by reducing the corresponding network parameters through the optimal scale, the prediction sub-models were obtained; finally, these sub-models are fused into a final prediction model with certain weight. The present study applies MSF dropout to carry out the experiments in MNIST and CIFAR-10 standard datasets. The result of the study shows that the prediction accuracy is significantly improved compared with the other two kinds of dropout, which verifies the effectiveness of the multi-scale fusion method.