Abstract
Although convolutional neural networks (CNNs) are widely used in image classification tasks and have demonstrated promising classification accuracy results, designing a CNN architecture requires a manual adjustment of parameters through a series of experiments as well as sufficient knowledge both in the problem domain and CNN architecture design. Therefore, it is difficult for users without prior experience to design a CNN for specific purposes. In this paper, we propose a framework for the automatic construction of CNN architectures based on ResNet, DenseNet, and Inception blocks and the roulette wheel selection method with a dynamic learning rate. Compared with the state of the art, the proposed approach has a significant improvement in the domain of image classification. Experimental evaluation of our approach including a comparison with the previous works on three benchmark datasets demonstrates the effectiveness of the overall method. The proposed algorithm not only improves the previous algorithm but also keeps the advantages of automatic CNN construction without requiring manual interventions. The source code of the our framework can be found at https://github.com/btogzhan2000/ea-cnn-complab.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Sun Y, Xue B, Zhang M, Yen GG (2020) Completely automated CNN architecture design based on blocks. IEEE Trans Neural Netw Learn Syst 31(4):1242–1254
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition (CVPR) 2015, pp 1–9
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition (CVPR) 2016, pp 770–778
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE conference on computer vision and pattern recognition (CVPR) 2017, pp 2261–2269
Zoph B, Le QV (2016) Neural architecture search with reinforcement learning. CoRR, vol. abs/1611.01578 [Online]. arXiv:1611.01578
Xie L, Yuille A (2017) Genetic CNN. In: IEEE international conference on computer vision (ICCV) 2017, pp 1388–1397
Zhong JYZ, Liu C-L (2018) Practical network blocks design with q-learning. In: Proceedings of the 2018 AAAI conference on artificial intelligence
Real E, Moore S, Selle A, Saxena S, Suematsu YL, Le QV, Kurakin A (2017) Large-scale evolution of image classifiers. CoRR vol. abs/1703.01041 [Online]. arXiv:1703.01041
Baker NNB, Gupta O, Raskar R (2017) Designing neural network architectures using reinforcement learning. In: Proceedings of the 2017 international conference on learning representations
Mnih V, Kavukcuoglu K, Silver D, Rusu A, Veness J, Bellemare M, Graves A, Riedmiller M, Fidjeland A, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 5187(540):529–533. https://doi.org/10.1038/nature14236
Lu Z, Whalen I, Dhebar YD, Deb K, Goodman ED, Banzhaf W, Boddeti VN (2019) Multi-criterion evolutionary design of deep convolutional neural networks. CoRR vol. abs/1912.01369 [Online]. arXiv:1912.01369
Londt T, Gao X, Andreae P (2020) Evolving character-level densenet architectures using genetic programming. CoRR, vol. abs/2012.02327 [Online]. arXiv:2012.02327
Sun Y, Xue B, Zhang M, Yen GG (2018) Automatically designing CNN architectures using genetic algorithm for image classification. CoRR, vol. abs/1808.03818 [Online]. arXiv:1808.03818
Barakbayeva T, Demirci MF (2021) Fully automatic CNN design with inception blocks. In: Jiang X, Fujita H (eds) Thirteenth international conference on digital image processing (ICDIP 2021), vol 11878, International Society for Optics and Photonics. SPIE, pp 275–280. https://doi.org/10.1117/12.2601117
Miller BL, Miller BL, Goldberg DE, Goldberg DE (1995) Genetic algorithms, tournament selection, and the effects of noise. Complex Syst 9:193–212
Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Teh YW, Titterington M (eds) Proceedings of the thirteenth international conference on artificial intelligence and statistics, series proceedings of machine learning research, vol 9. Chia Laguna Resort, Sardinia, Italy: PMLR, 13–15 May 2010, pp 249–256. http://proceedings.mlr.press/v9/glorot10a.html
Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Krizhevsky A (2009) Learning multiple layers of features from tiny images. Technical report
Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition 2009, pp 248–255
Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng A (2011) Reading digits in natural images with unsupervised feature learning. NIPS, 01
Zeiler M (2012) Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701
Luo H, Hanagud S (1997) Dynamic learning rate neural network training and composite structural damage detection. AIAA J 35(9):1522–1527
Larsson MMG, Shakhnarovich G (2016) Fractalnet: ultradeep neural networks without residuals. In: The 5th international conference on learning representations
Acknowledgements
This research has been funded under the Nazarbayev University faculty development grant “Forming Reliable Feature Correspondences and Distortion-free Graph Embedding with Deep Learning”. Grant# 110119FD4530.
Funding
M. Fatih Demirci has received research grants from Nazarbayev University (Kazakhstan) and TUBITAK (Turkey), and has worked at Nazarbayev University (Kazakhstan), TOBB University of Economics and Technology (Turkey), Utrecht University (Netherlands), and Drexel University (USA) before.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Togzhan Barakbayeva declares no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Barakbayeva, T., Demirci, F.M. Fully automatic CNN design with inception and ResNet blocks. Neural Comput & Applic 35, 1569–1580 (2023). https://doi.org/10.1007/s00521-022-07700-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-022-07700-9