Abstract
The performance of deep neural networks (DNNs) often depends on the design of their architectures. But designing a DNN with good performance is a difficult and knowledge-intensive process. In this paper, we propose a neural architecture search method based on improved brain storm optimization (BSO) algorithm to efficiently deal with image classification tasks. BSO successfully transposes the human brainstorming process to design of optimization algorithms, which typically uses grouping, substitution, and creation operators to generate as many solutions as possible to approach the global optimization of the problem generation by generation. However, the BSO algorithm using clustering methods for grouping increases the computational burden, so we use the BSO algorithm based on simple grouping methods to solve the optimal architecture of the neural architecture search (NAS). We also redesigned the search space and designed an efficient encoding strategy for each individual.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Li, N., Ma, L., Yu, G., et al.: Survey on evolutionary deep learning: principles, algorithms, applications and open issues. arXiv preprint arXiv:2208.10658 (2022)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. IEEE (2016)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Zhang, Y., Chan, W., Jaitly, N.: Very deep convolutional networks for end-to-end speech recognition. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4845–4849. IEEE (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016)
Ma, L., Li, N., Yu, G., et al.: How to simplify search: classification-wise Pareto evolution for one-shot neural architecture search. arXiv preprint arXiv:2109.07582 (2021)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)
Liu, H., Simonyan, K., Yang, Y.: Darts: differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018)
Bäck, T., Fogel, D.B., Michalewicz, Z.: Handbook of evolutionary computation. Release 97(1), B1 (1997)
Ma, L., Wang, X., Huang, M., et al.: A novel evolutionary root system growth algorithm for solving multi-objective optimization problems. Appl. Soft Comput. 57, 379–398 (2017)
Sun, Y., Xue, B., Zhang, M., Yen, G.G.: Evolving deep convolutional neural networks for image classifification. IEEE Trans. Evol. Comput. 24(2), 394–407 (2020)
Sun, Y., Xue, B., Zhang, M., Yen, G.G.: Completely automated CNN architecture design based on 9 blocks. IEEE Trans. Neural Netw. Learn. Syst. 31(4), 1242–1254 (2020)
Mitchell, M.: An Introduction to Genetic Algorithms. MIT Press, Cambridge (1998)
Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming: An Introduction: On the Automatic Evolution of Computer Programs and its Applications. Morgan Kaufmann Publishers Inc. (1998)
Das, S., Suganthan, P.N.: Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15(1), 4–31 (2011)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN 1995 International Conference on Neural Networks, vol. 4, pp. 1942–1948. IEEE (1995)
Tuba, E., Strumberger, I.: Classification and feature selection method for medical datasets by brain storm optimization algorithm and support vector machine. Procedia Comput. Sci. 162, 307–315 (2019)
Ma, L., Wang, X., Huang, M., Lin, Z., Tian, L., Chen, H.: Two-level master-slave RFID networks planning via hybrid multiobjective artificial bee colony optimizer. IEEE Trans. Syst. Man Cybern.: Syst. 49(5), 861–880 (2019). https://doi.org/10.1109/TSMC.2017.2723483
Ma, L., Hu, K., Zhu, Y., et al.: Discrete and continuous optimization based on hierarchical artificial bee colony optimizer. J. Appl. Math. 2014, 1–20 (2014). https://doi.org/10.1155/2014/402616
Chen, H., Zhu, Y., Hu, K., et al.: Bacterial colony foraging algorithm: combining chemotaxis, cell-to-cell communication, and self-adaptive strategy. Inf. Sci. 273, 73–100 (2014)
Ma, L., Li, N., Guo, Y., et al.: Learning to optimize: reference vector reinforcement learning adaption to constrained many-objective optimization of industrial copper burdening system. IEEE Trans. Cybern. 52(12), 12698–12711 (2022). https://doi.org/10.1109/TCYB.2021.3086501
Zhang, B., Wang, X., Ma, L., Huang, M.: Optimal controller placement problem in internet-oriented software defined network. In: 2016 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), Chengdu, China, pp. 481–488 (2016). https://doi.org/10.1109/CyberC.2016.98
Sun, Y., Yen, G.G., Yi, Z.: IGD indicator-based evolutionary algorithm for many-objective optimization problems. IEEE Trans. Evol. Comput. 23(2), 173–187 (2018)
Xue, Y., Zhao, Y.: Structure and weights search for classification with feature selection based on brain storm optimization algorithm. Appl. Intell. 52(5), 5857–5866 (2022)
Zeng, R., Su, M., Yu, R., Wang, X.: CD2: fine-grained 3D mesh reconstruction with twice chamfer distance. ACM Trans. Multimed. Comput. Commun. Appl. Just Accepted (2023). https://doi.org/10.1145/3582694
Cheng, S., Zhang, M., Ma, L., Lu, H., Wang, R., Shi, Y.: Brain storm optimization algorithm for solving knowledge spillover problems. Neural Comput. Appl. 35, 12247–12260 (2021). https://doi.org/10.1007/s00521-020-05674-0
Ma, L., Cheng, S., Shi, Y.: Enhancing learning efficiency of brain storm optimization via orthogonal learning design. IEEE Trans. Syst. Man Cybern.: Syst. 51(11), 6723–6742 (2020)
Tran, B., Xue, B., Zhang, M.: A new representation in PSO for discretization-based feature selection. IEEE Trans. Cybern. 48(6), 1733–1746 (2017)
Zhan, Z., Zhang, J., Shi, Y., et al.: A modified brain storm optimization. In: 2012 IEEE Congress on Evolutionary Computation, pp. 1–8. IEEE (2012)
Cheng, S.,Shi, Y., Qin, Q., et al.: Solution clustering analysis in brain storm optimization algorithm. In: Proceedings of the 2013 IEEE Symposium on Swarm Intelligence (SIS 2013), Singapore, pp. 111–118. IEEE (2013)
Sun, Y., Sun, X., Fang, Y., et al.: A novel training protocol for performance predictors of evolutionary neural architecture search algorithms. IEEE Trans. Evol. Comput. 25(3), 524–536 (2021)
Shi, Y.: Brain storm optimization algorithm. In: Tan, Y., Shi, Y., Chai, Yi., Wang, G. (eds.) Advances in Swarm Intelligence, pp. 303–309. Springerg, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21515-5_36
Xue, B., Zhang, M., Browne, W.N.: Particle swarm optimization for feature selection in classification: novel initialisation and updating mechanisms. Appl. Soft Comput. J. 18, 261–276 (2014)
Xie, L., Yuille, A.: Genetic CNN. In: Proceedings of the IEEE International Conference on ComputerVision (ICCV), Venice, Italy, pp. 1388–1397 (2017)
Cai, H., Chen, T., Zhang, W., Yu, Y., Wang, J.: Efficient architecturesearch by network transformation. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, pp. 1–8 (2018)
Real, E., et al.: Large-scale evolution of image classifiers. In: Proceedings of the 34thInternational Conference on Machine Learning, pp. 2902–2911 (2017)
Cai, H., Gan, C., Wang, T., Zhang, Z., Han, S.: Once for all: train one network and specialize it for efficient deployment. In: International Conference on Learning Representations (2020)
Trofimov, I., Klyuchnikov, N., Salnikov, M., Filippov, A., Burnaev, E.: Multi-fidelity neural architecture search with knowledge distillation. IEEE Access. https://doi.org/10.1109/ACCESS.2023.3234810
Xie, X., Liu, Y., Sun, Y., et al.: Benchenas: a benchmarking platform for evolutionary neural architecture search. IEEE Trans. Evol. Comput. 26(6), 1473–1485 (2022)
Acknowledgments
This work was supported in part by the Fundamental Research Funds for the Central Universities No. N2117005, the Joint Funds of the Natural Science Foundation of Liaoning Province und Grant 2021-KF-11-01 and the Fundamental Research Funds for the Central Universities.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
An, X. et al. (2023). Neural Architecture Search Based on Improved Brain Storm Optimization Algorithm. In: Tan, Y., Shi, Y., Luo, W. (eds) Advances in Swarm Intelligence. ICSI 2023. Lecture Notes in Computer Science, vol 13968. Springer, Cham. https://doi.org/10.1007/978-3-031-36622-2_27
Download citation
DOI: https://doi.org/10.1007/978-3-031-36622-2_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36621-5
Online ISBN: 978-3-031-36622-2
eBook Packages: Computer ScienceComputer Science (R0)