Ricardo H. R. Lima
ABSTRACT
The use of Convolutional Neural Networks (CNNs) have proven to be a solid approach often used to solve many Machine Learning problems, such as image classification and natural language processing tasks. The manual design of CNNs is a hard task, due to the high number of configurable parameters and possible configurations. Recent studies around the automatic design of CNNs have shown positive results. In this work, we propose to explore the design of CNN architectures through the use of Grammatical Evolution (GE), where a BNF grammar is used to define the CNN components and structural rules. Experiments were performed using the MNIST and CIFAR-10 datasets. The obtained results show that the presented approach achieved competitive results and maintaining relatively small architectures when compared with similar state-of-the-art approaches.
- James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of Machine Learning Research 13, Feb (2012), 281--305. Google Scholar
Digital Library
- James Bergstra, Daniel Yamins, and David Daniel Cox. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. (2013).Google Scholar
- Tsung-Han Chan, Kui Jia, Shenghua Gao, Jiwen Lu, Zinan Zeng, and Yi Ma. 2015. PCANet: A simple deep learning baseline for image classification? IEEE Transactions on Image Processing 24, 12 (2015), 5017--5032.Google ScholarDigital Library
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. Technical Report. Citeseer.Google Scholar
- Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.Google ScholarCross Ref
- Ilya Loshchilov and Frank Hutter. 2016. CMA-ES for hyperparameter optimization of deep neural networks. arXiv preprint arXiv:1604.07269 (2016).Google Scholar
- Risto Miikkulainen, Jason Liang, Elliot Meyerson, Aditya Rawal, Daniel Fink, Olivier Francon, Bala Raju, Hormoz Shahrzad, Arshak Navruzyan, Nigel Duffy, et al. 2019. Evolving deep neural networks. In Artificial Intelligence in the Age of Neural Networks and Brain Computing. Elsevier, 293--312.Google Scholar
- Masanori Suganuma, Shinichi Shirakawa, and Tomoharu Nagao. 2017. A genetic programming approach to designing convolutional neural network architectures. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM, 497--504. Google ScholarDigital Library
- Yanan Sun, Bing Xue, and Mengjie Zhang. 2017. Evolving deep convolutional neural networks for image classification. arXiv preprint arXiv:1710.10741 (2017).Google Scholar
- Yanan Sun, Gary G Yen, and Zhang Yi. 2018. Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations. IEEE Transactions on Evolutionary Computation (2018).Google Scholar
Index Terms
- Evolving convolutional neural networks through grammatical evolution
Recommendations
A new grammatical evolution method for generating deep convolutional neural networks with novel topologies
GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference CompanionNeuroevolution, a sub-field of AutoML, utilizes evolutionary algorithms to automate the process of creating Deep Neural Networks architectures. For problems with complex objects, such as neural networks, Grammar-based Evolutionary Algorithms (GEs) can be ...
Towards dropout training for convolutional neural networks
Recently, dropout has seen increasing use in deep learning. For deep convolutional neural networks, dropout is known to work well in fully-connected layers. However, its effect in convolutional and pooling layers is still not clear. This paper ...
Efficient densely connected convolutional neural networks
Highlights- Proposed two efficient densely connected ConvNets, DesneDsc and Dense2Net.
- ...
Graphical abstractDisplay Omitted
AbstractRecent works have shown that convolutional neural networks (CNNs) are parameter redundant, which limits the application of CNNs in Mobile devices with limited memory and computational resources. In this paper, two novel and efficient ...
Comments