Abstract
Convolutional neural networks (CNNs) can learn the features of samples by supervised manner, and obtain outstanding achievements in many application fields. In order to improve the performance and generalization of CNNs, we propose a self-learning hybrid dilated convolution neural network (SPHDCNN), which can choose relatively reliable samples according to the current learning ability during training. In order to avoid the loss of useful feature map information caused by pooling, we introduce hybrid dilated convolution. In the proposed SPHDCNN, weight is applied to each sample to reflect the easiness of the sample. SPHDCNN employs easier samples for training first, and then adds more difficulty samples gradually according to the current learning ability. It gradually improves its performance by this learning mechanism. Experimental results show SPHDCNN has strong generalization ability, and it achieves more advanced performance compared to the baseline method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Basu S, Christensen J (2013) Teaching classification boundaries to humans. In: Twenty-Seventh AAAI Conference on Artificial Intelligence
Bengio Y, Louradour J, Collobert R, Weston J (2009) Curriculum learning. In: Proceedings of the 26th annual international conference on machine learning, pp 41–48
Fang Y, Li Y, Tu X, Tan T, Wang X (2020) Face completion with hybrid dilated convolution. Signal Process Image Commun 80:115664
Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2414–2423
Jiao C, Wang X, Gou S, Chen W, Li D, Chen C, Li X (2019) Self-paced convolutional neural network for polsar images classification. Remote Sens 11(4):424
Khan F, Mutlu B, Zhu J (2011) How do humans teach: On curriculum learning and teaching dimension. In: Advances in neural information processing systems, pp 1449–1457
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980
Kumar MP, Packer B, Koller D (2010) Self-paced learning for latent variable models. In: Advances in Neural Information Processing Systems, pp 1189–1197
Larochelle H, Erhan D, Courville A, Bergstra J, Bengio Y (2007) An empirical evaluation of deep architectures on problems with many factors of variation. In: Proceedings of the 24th international conference on Machine learning, pp 473–480
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Li H, Gong M (2017) Self-paced convolutional neural networks.. In: IJCAI, pp 2110–2116
Li H, Gong M, Wang C, Miao Q (2018) Self-paced stacked denoising autoencoders based on differential evolution for change detection. Appl Soft Comput 71:698–714
Liu C, Shang Z, Qin A (2019) A multiscale image denoising algorithm based on dilated residual convolution network. In: Chinese Conference on Image and Graphics Technologies, Springer, pp 193–203
Lu Z, Yu Z, Yali P, Shigang L, Xiaojun W, Gang L, Yuan R (2018) Fast single image super-resolution via dilated residual networks. IEEE Access 7:109729–109738
Meng D, Zhao Q, Jiang L (2015) What objective does self-paced learning indeed optimize?. arXiv:1511.06049
Oord A, Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, Kalchbrenner N, Senior A, Kavukcuoglu K (2016) Wavenet: A generative model for raw audio. arXiv:1609.03499
Paoletti ME, Haut JM, Plaza J, Plaza A (2018) A new deep convolutional neural network for fast hyperspectral image classification. ISPRS journal of photogrammetry and remote sensing 145:120–147
Radovic M, Adarkwa O, Wang Q (2017) Object recognition in aerial images using convolutional neural networks. Journal of Imaging 3(2):21
Rifai S, Vincent P, Muller X, Glorot X, Bengio Y (2011) Contractive auto-encoders: Explicit invariance during feature extraction
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15(1):1929–1958
Tang K, Ramanathan V, Fei-Fei L, Koller D (2012) Shifting weights: Adapting object detectors from image to video. In: Advances in Neural Information Processing Systems, pp 638–646
Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE winter conference on applications of computer vision (WACV), IEEE, pp 1451–1460
Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122
Zhao Q, Meng D, Jiang L, Xie Q, Xu Z, Hauptmann AG (2015) Self-paced learning for matrix factorization. In: Twenty-ninth AAAI conference on artificial intelligence
Zhu P, Hao C, Hu Q, Wang Q, Zhang C (2018) Towards generalized and efficient metric learning on riemannian manifold. In: Twenty-Seventh International Joint Conference on Artificial Intelligence IJCAI-18
Zhu P, Ren Q, Hu Q, Wang Q, Liu Y (2018) Beyond similar and dissimilar relations : A kernel regression formulation for metric learning. In: Twenty-Seventh International Joint Conference on Artificial Intelligence IJCAI-18
Zhu X, Zhu Y, Zheng W (2019) Spectral rotation for deep one-step clustering, Pattern Recogn, https://doi.org/10.1016/j.patcog.2019.107175
Zhu X, Gan J, Lu G, Li J, Zhang S (2020) Spectral clustering via half-quadratic optimization. World Wide Web 23:1969–1988
Zöhrer M, Pernkopf F (2014) General stochastic networks for classification. In: Advances in Neural Information Processing Systems, pp 2015–2023
Acknowledgments
This work was supported by the National Natural Science Foundation of China (Grant No: 61836016, 61876046, 81701780 and 61672177); the Guangxi Natural Science Foundation (Grant No: 2017GXNSFBA198221); the Project of Guangxi Science and Technology (GuiKeAD19110133 and GuiKeAD17195062); Innovation Project of Guangxi Graduate Education (Grant No: JXXYYJSCXXM-012, JXXYYJSCXXM-011, JGY20200026); Research and Education Project of Guangxi Normal University (Grant No: 2019YR006); Research Fund of Guangxi Key Lab of Multi-source Information Mining & Security (Grant No: 20-A-01-01); and Basic Competence Promotion Project for Young and Middle-aged Teachers in Guangxi Education Department (Grant No: 2019KY0062).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, W., Lu, G., Zhang, S. et al. Self-paced hybrid dilated convolutional neural networks. Multimed Tools Appl 81, 34169–34181 (2022). https://doi.org/10.1007/s11042-020-09868-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09868-5