EMaxPPE: Epoch’s Maximum Prediction Probability Ensemble Method for Deep Learning Classification Models

Abstract

As deep learning (DL) is evolving rapidly, implementing the knowledge of DL into various fields of human life and the effective usage of existing data insights are becoming crucial tasks for a majority of DL models. We are proposing to ensemble maximum prediction probabilities of different epochs and the epoch which achieved the highest accuracy for classification problems. Our suggestion contributes to the improvement of DL models using the pre-trained and skipped results from epochs. The maximum prediction probability ensemble of epochs increases the prediction space of the entire model if the intersection of prediction scope of any epoch is smaller than the one that has the biggest prediction scope. Using only the best epoch’s prediction probabilities for classification cannot use the other epochs’ knowledge. To avoid bias in this research, a simple CNN architecture with batch normalization and dropout was used as a base model. By ensembling only maximum prediction probabilities of different epochs, we managed to use 50% of the lost data insight from the epochs, thereby increasing the total accuracy by 4–5%.