Low-Shot Learning and Pattern Separation using Cellular Automata Integrated CNNs
Article No.: 17, Pages 1 - 9
Abstract
Traditionally, deep convolutional neural networks are computationally expensive to train and require large amounts of data samples. In this article, we explore the use of pre-trained cellular automata as a substitute for convolutional layers. We propose a specialized cellular automata system with multilayered attributes and kernels to better harness its inherent spatio-temporal processing capabilities. An architecture search, combining deep Q-learning with a speciated genetic algorithm, is used to optimize the kernels and identify cell attributes. Experiments under low-shot conditions demonstrate that the cellular automata-integrated CNN outperforms compact state-of-the-art CNN models by 6-10% on static image datasets and 8-12% on temporal image sequence datasets.
References
[1]
AI Adamatzky. 1996. Computation of shortest path in cellular automata. Mathematical and Computer Modelling 23, 4 (1996), 105–113.
[2]
Stefania Bandini, Giancarlo Mauri, and Roberto Serra. 2001. Cellular automata: From a theoretical parallel computational model to its application to complex systems. Parallel Comput. 27, 5 (2001), 539–553.
[3]
Stefania Bandini, Leonardo Vanneschi, Andrew Wuensche, and Alessandro Bahgat Shehata. 2009. Cellular Automata Pattern Recognition and Rule Evolution Through a Neuro-Genetic Approach.J. Cell. Autom. 4, 3 (2009), 171–181.
[4]
Dave Burraston and Ernest Edmonds. 2005. Cellular automata in generative electronic music and sonic art: a historical and technical review. Digital Creativity 16, 3 (2005), 165–185.
[5]
Chun-Ling Chang, Yun-Jie Zhang, and Yun-Yin Gdong. 2004. Cellular automata for edge detection of images. In Proceedings of 2004 international conference on machine learning and cybernetics (IEEE Cat. No. 04EX826), Vol. 6. IEEE, 3830–3834.
[6]
Edgar F Codd. 2014. Cellular automata. Academic press.
[7]
Antonio Della Cioppa, Angelo Marcelli, and Prisco Napoli. 2011. Speciation in evolutionary algorithms: adaptive species discovery. In Proceedings of the 13th annual conference on Genetic and evolutionary computation. 1053–1060.
[8]
Li Deng. 2012. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine 29, 6 (2012), 141–142.
[9]
Francis Ferraro, Nasrin Mostafazadeh, Ting-Hao (Kenneth) Huang, Lucy Vanderwende, Jacob Devlin, Michel Galley, and Margaret Mitchell. 2015. On Available Corpora for Empirical Methods in Vision & Language. CoRR abs/1506.06833(2015). arXiv:1506.06833http://arxiv.org/abs/1506.06833
[10]
Niloy Ganguly, Biplab K Sikdar, Andreas Deutsch, Geoffrey Canright, and P Pal Chaudhuri. 2003. A survey on cellular automata. (2003).
[11]
Daniele Grattarola, Lorenzo Livi, and Cesare Alippi. 2021. Learning graph cellular automata. Advances in Neural Information Processing Systems 34 (2021).
[12]
Kaiming He and Jian Sun. 2015. Convolutional neural networks at constrained time cost. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5353–5360.
[13]
KGF Janssens. 2010. An introductory review of cellular automata modeling of moving grain boundaries in polycrystalline materials. Mathematics and Computers in Simulation 80, 7 (2010), 1361–1381.
[14]
Niels Justesen, Ruben Rodriguez Torrado, Philip Bontrager, Ahmed Khalifa, Julian Togelius, and Sebastian Risi. 2018. Illuminating generalization in deep reinforcement learning through procedural level generation. arXiv preprint arXiv:1806.10729(2018).
[15]
Daniel Justus, John Brennan, Stephen Bonner, and Andrew Stephen McGough. 2018. Predicting the computational cost of deep learning models. In 2018 IEEE international conference on big data (Big Data). IEEE, 3873–3882.
[16]
HM Kabir, Moloud Abdar, Seyed Mohammad Jafar Jalali, Abbas Khosravi, Amir F Atiya, Saeid Nahavandi, and Dipti Srinivasan. 2020. Spinalnet: Deep neural network with gradual input. arXiv preprint arXiv:2007.03347(2020).
[17]
Tapas Kumar and G Sahoo. 2010. A novel method of edge detection using cellular automata. International Journal of Computer Applications 9, 4(2010), 38–44.
[18]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436–444.
[19]
Zewen Li, Fan Liu, Wenjie Yang, Shouheng Peng, and Jun Zhou. 2021. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems (2021).
[20]
Ming Liang and Xiaolin Hu. 2015. Recurrent Convolutional Neural Network for Object Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[21]
Jialin Liu, Sam Snodgrass, Ahmed Khalifa, Sebastian Risi, Georgios N Yannakakis, and Julian Togelius. 2021. Deep learning for procedural content generation. Neural Computing and Applications 33, 1 (2021), 19–37.
[22]
Gisele Helena Barboni Miranda, Jeaneth Machicao, and Odemir Martinez Bruno. 2016. Exploring spatio-temporal dynamics of cellular automata for pattern recognition in networks. Scientific Reports 6, 1 (2016), 1–15.
[23]
Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson, and Michael Levin. 2020. Growing neural cellular automata. Distill 5, 2 (2020), e23.
[24]
Stefano Nichele and Andreas Molund. 2017. Deep learning with cellular automaton-based reservoir computing. (2017).
[25]
Stefano Nichele, Mathias Berild Ose, Sebastian Risi, and Gunnar Tufte. 2017. CA-NEAT: evolved compositional pattern producing networks for cellular automata morphogenesis and replication. IEEE Transactions on Cognitive and Developmental Systems 10, 3(2017), 687–700.
[26]
Ettore Randazzo, Alexander Mordvintsev, Eyvind Niklasson, Michael Levin, and Sam Greydanus. 2020. Self-classifying mnist digits. Distill 5, 8 (2020), e00027–002.
[27]
Sebastian Risi and Julian Togelius. 2020. Increasing generality in machine learning through procedural content generation. Nature Machine Intelligence 2, 8 (2020), 428–436.
[28]
Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. 2015. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems 28 (2015).
[29]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556(2014).
[30]
Shyam Sudhakaran, Djordje Grbic, Siyan Li, Adam Katona, Elias Najarro, Claire Glanois, and Sebastian Risi. 2021. Growing 3D Artefacts and Functional Machines with Neural Cellular Automata. CoRR abs/2103.08737(2021). arXiv:2103.08737https://arxiv.org/abs/2103.08737
[31]
Giuseppe A Trunfio. 2004. Predicting wildfire spreading through a hexagonal cellular automata model. In International Conference on Cellular Automata. Springer, 385–394.
[32]
Michail-Antisthenis I Tsompanas and Georgios Ch Sirakoulis. 2012. Modeling and hardware implementation of an amoeba-like cellular automaton. Bioinspiration & Biomimetics 7, 3 (2012), 036013.
[33]
Lynette Van Zijl and Leendert Botha. 2009. Feature Extraction for Image Pattern Matching with Cellular Automata. In Stringology. 3–14.
[34]
Kathryn Walker, Rasmus Berg Palm, Rodrigo Moreno Garcia, Andres Faina, Kasper Stoy, and Sebastian Risi. 2022. Physical Neural Cellular Automata for 2D Shape Classification. arXiv preprint arXiv:2203.07548(2022).
[35]
Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. CoRR abs/1708.07747(2017). arXiv:1708.07747http://arxiv.org/abs/1708.07747
[36]
Dongsu Zhang, Changwoon Choi, Jeonghwan Kim, and Young Min Kim. 2021. Learning to generate 3d shapes with generative cellular automata. arXiv preprint arXiv:2103.04130(2021).
Index Terms
- Low-Shot Learning and Pattern Separation using Cellular Automata Integrated CNNs
Recommendations
Descriptional complexity of cellular automata and decidability questions
Third international workshop on descriptional complexity of automata, grammars and related structuresWe study the descriptional complexity of cellular automata (CA) which are a parallel model of computation. We show that between one of the simplest cellular models, the realtime one-way CA (realtime-OCA), and "classical" models like deterministic finite ...
Closure properties of cellular automata
Concerning the power of one-dimensional cellular automata recognizers, Ibarra and Jiang have proved that real time cellular automata (CA) and linear time CA are equivalent if and only if real time CA is closed under reverse. In this paper we investigate ...
Simulations Between Multi-dimensional Deterministic and Alternating Cellular Automata
Cellular AutomataWe present simulation and separation results between multi-dimensional deterministic and alternating cellular automata (CAs). It is shown that for any integers k≥l≥1, every k-dimensional t(n)-time deterministic CA can be simulated by an l-dimensional O(...
Comments
Information & Contributors
Information
Published In
July 2022
213 pages
ISBN:9781450397896
DOI:10.1145/3546790
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 September 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICONS
Acceptance Rates
Overall Acceptance Rate 13 of 22 submissions, 59%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 87Total Downloads
- Downloads (Last 12 months)15
- Downloads (Last 6 weeks)7
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format