skip to main content
10.1145/3331453.3361326acmotherconferencesArticle/Chapter ViewAbstractPublication PagescsaeConference Proceedingsconference-collections
research-article

Application of Metaheuristic Algorithms for Determining the Structure of a Convolutional Neural Network with a Small Dataset

Published: 22 October 2019 Publication History

Abstract

Inadequately labeled data can limit the accuracy of classification in image recognition tasks. Several methods have been proposed in the past to alleviate this limitation, such as transfer learning and data augmentation. However, the classification accuracy of the convolutional neural network (CNN) largely depends on its structural parameters, which are known as hyper-parameters. Therefore, in this paper, we introduce another method for minimizing the misclassification rate in a given small dataset by determining the hyper-parameters. The harmony search (HS) algorithm, improved harmony search (IHS) algorithm, self-adaptive global best harmony search (SGHS) algorithm, and novel global harmony search (NGHS) algorithm are applied for determining the optimal hyper-parameters. Additionally, we also compared the estimation performances of these four HS algorithms. It was finally observed that the HS and the IHS algorithms greatly outperform the other two algorithms.

References

[1]
K Simonyan and A Zisserman (2015). Very deep convolutional networks for large-scale Image Recognition. ICLR, 1--14.
[2]
A Krizhevsky, I Sutskever and G E Hinton (2012). ImageNet classification with deep convolutional neural networks. NIPS, 1106--1114.
[3]
R Girshick, J Donahue, T Darrell, U C Berkeley and J Malik (2015). Rich feature hierarchies for accurate object detection and semantic segmentation. CVPR, 580--587, June 2014.
[4]
J Redmon, S Divvala, R Girshick and A Farhadi (2016). You only look once: unified, real-Time object detection. CVPR, 779--788.
[5]
F Sung, Y Yang and L Zhang (2018). Learning to compare: relation network for few-shot learning. CVPR, 1199--1208.
[6]
L Perez and J Wang (2017). The Effectiveness of Data Augmentation in Image Classification using Deep Learning, http://arxiv.org/abs/1712.04621.
[7]
R Takahashi, T Matsubara and K Uehara (2015). Data augmentation using random image cropping and patching for deep CNNs. Journal of Latex class Files, 14(8), 1--11.
[8]
S Ravi and H Larochelle (2017). Optimization as a model for few-short learning. ICLR, Toulon, France, Apr. 2017.
[9]
R Caruana (1995). Learning many related tasks at the same time with back-propagation. Adv. Neural Inf. Process, 657--664, July 1995.
[10]
Y Bengio (2012). Deep learning of representations for unsupervised and transfer Learning. Workshop and Conference Proceedings, 27, 17--37.
[11]
J Donahue, Y Jia, O Vinyals, J Hoffman, N Zhang, E Tzeng and T Darrell (2014). DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. Proceedings of the 31st International Conference on Machine Learning, Beijing, China, 2014.
[12]
J Yosinski, J Clune, Y Bengio and H Lipson (2014). How transferable are features in deep neural networks?. NIPS, Montreal, Canada, 2014.
[13]
R Wagner, M Thom, R Schweiger, G Palm and A Rothermel (2013). Learning convolutional neural networks from few samples. Proceedings of International Joint Conference on Neural Networks, 1884--1890, Aug. 2013.
[14]
S J Pan and Q Yang (2010). A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10), 1345--1359.
[15]
V Bibaeva (2018). Using metaheuristic for hyper-parameters optimization of convolutional neural network. 2018 IEEE 28th Int. Work. Mach. Learn. Signal Process, Aalborg, Denmark, 2018.
[16]
X S Yang (2010). Engineering optimization: an introduction with metaheuristic application. John Wiley & Sons, Hoboken, NJ, USA, 2010.
[17]
Z W Geem, J H Kim and G V Loganathan (2001). A New Heuristic Optimization Algorithm: Harmony Search. Simulation (2001), 76, 60--68.
[18]
S Tavakoli, E Valian, S Mohanna (2012). Feedforward neural network training using intelligent global harmony search. Evol. Syst., 3(2), 125--131.
[19]
G Rose, J Papa, A Marana, W Scheirer and D Cox (2015). Fine-tuning convolutional neural networks using harmony search. Iberoamerican Congress on Pattern Recognition, Montevideo, Uruguay, 2015.
[20]
L M Rasdi Rere, M I Fanany and A M Arymurthy (2016). Metaheuristic algorithms for convolution neural network. Computational Intelligence and Neuroscience, 2016, 1--13.
[21]
K Fukushima (1988). Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural Networks (1988), 1(2), 119--130.
[22]
Y LeCun, L Bottou, Y Bengio and P Haffner (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278--2324.
[23]
K S Lee and Z W Geem (2005). A new meta-heuristic algorithm for continuous engineering optimization: harmony search theory and practice. Comput. Method Appl. Mech. Eng., 194, 3902--3933.
[24]
M Mahdavi, M Fesanghary and E Damangir (2007). An improved harmony search algorithm for solving optimization problems. Appl. Math. Comput, 188, 1567--1579.
[25]
Q K Pan, P N Suganthan, M F Tasgetiren and J J Liang (2010). A self-adaptive global best harmony search algorithm for continuous optimization problems. Appl. Math. Comput. (2010), 216, 830--848.
[26]
C Y Chiu, P C Shih and X Li (2018). A dynamic adjusting novel global harmony search for continuous optimization problems. Symmetry (Basel), 10(337), 1--31.
[27]
D Zou, L Gao, S Li, J Wu and X Wang (2010). A novel global harmony search algorithm for task assignment problem. J. Syst. Softw, 83, 1678--1688.
[28]
D E Goldberg (1989). Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Boston, MA, 1989.
[29]
M Gonen (2007). Analyzing receiver operating characteristic curves with SAS. SAS Institute Inc. (2007), Cary, NC, USA, 2007.
[30]
E Valian, S Tavakoli and S Mohanna (2014). An intelligent global harmony search approach to continuous optimization problems. Appl. Math. Comput. Applied Mathematics and Computation, 232, 670--684.
[31]
X S Yang, M Karamanoglu and X He (2014). Flower pollination algorithm: a novel approach for multiobjective optimization. Engineering Optimization, 46(9), 1222--1237.
[32]
S Mirjalili and A Lewis (2016). The whale optimization algorithm. Advances in Engineering Software, 95 (2016), 51--67.

Cited By

View all
  • (2021)Advanced Meta-Heuristics, Convolutional Neural Networks, and Feature Selectors for Efficient COVID-19 X-Ray Chest Image ClassificationIEEE Access10.1109/ACCESS.2021.30610589(36019-36037)Online publication date: 2021

Index Terms

  1. Application of Metaheuristic Algorithms for Determining the Structure of a Convolutional Neural Network with a Small Dataset

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    CSAE '19: Proceedings of the 3rd International Conference on Computer Science and Application Engineering
    October 2019
    942 pages
    ISBN:9781450362948
    DOI:10.1145/3331453
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 22 October 2019

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Convolutional neural network
    2. Harmony search algorithms
    3. Metaheuristic Algorithms
    4. Structure

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    CSAE 2019

    Acceptance Rates

    Overall Acceptance Rate 368 of 770 submissions, 48%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)3
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 19 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2021)Advanced Meta-Heuristics, Convolutional Neural Networks, and Feature Selectors for Efficient COVID-19 X-Ray Chest Image ClassificationIEEE Access10.1109/ACCESS.2021.30610589(36019-36037)Online publication date: 2021

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media