Abstract
NSGA-Net is a popular method for neural architecture search (NAS). It conducts the improved non-dominated sorting genetic algorithm (NSGA-II) during its search procedure. In this paper, a NAS method using the multi-objective evolutionary algorithm based on decomposition (MOEA/D-Net) is proposed to heighten the running efficiency of NSGA-Net. MOEA/D-Net aims to minimize the number of floating-point operations (FLOPs) and error rate of neural architectures through the multi-objective evolutionary algorithm based on decomposition (MOEA/D) during the search process. It selects parents within the neighborhoods of a subproblem and conducts multi-point crossover and mutation to generate offspring individuals at every generation. Experiment results on the CIFAR-10 image classification dataset indicate that MOEA/D-Net obtains architecture networks with less FLOPs and MOEA/D-Net outperforms NSGA-Net in terms of running efficiency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9. IEEE (2015)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE (2016)
Huang, G., Liu, Z., Van Der Maaten L., et al.: Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708. IEEE (2017)
Howard, A. G., Zhu, M., Chen, B., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A.: XNOR-Net: ImageNet classification using binary convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 525–542. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_32
Courbariaux, M., Hubara, I., Soudry, D., et al.: Binarized neural networks: training deep neural networks with weights and activations constrained to +1 or -1. arXiv preprint arXiv:1602.02830 (2016)
Baker, B., Gupta, O., Naik, N., et al.: Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167 (2016)
Zhong, Z., Yan, J., Wu, W., et al.: Practical block-wise neural network architecture generation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2423–2432. IEEE (2018)
Zoph, B., Vasudevan, V., Shlens, J., et al.: Learning transferable architectures for scalable image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8697–8710. IEEE (2018)
Miconi, T.: Neural networks with differentiable structure. arXiv preprint arXiv:1606.06216 (2016)
Liu, H., Simonyan, K., Yang, Y.: DARTS: differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018)
Real, E., Moore, S., Selle, A., et al.: Large-scale evolution of image classifiers. In: International Conference on Machine Learning, pp. 2902–2911. PMLR (2017)
Xie, L., Yuille, A.: Genetic CNN. In: IEEE International Conference on Computer Vision, pp. 1379–1388. IEEE (2017)
Real, E., Aggarwal, A., Huang, Y., et al.: Regularized evolution for image classifier architecture search. In: AAAI Conference on Artificial Intelligence, pp. 4780–4789. AAAI (2019)
Chu, X., Zhang, B., Ma, H., et al.: Fast, accurate and lightweight super-resolution with neural architecture search. arXiv preprint arXiv:1901.07261 (2019)
Lu, Z., Whalen, I., Boddeti, V., et al.: NSGA-Net: a multi-objective genetic algorithm for neural architecture search. arXiv preprint arXiv:1810.03522 (2018)
Felix, J.-X., Naresh Boddeti, V., Savvides, M.: Local binary convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 19–28. IEEE (2017)
Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016)
DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017)
Acknowledgments
This work was supported in part by the Natural Science Foundation of Guangdong Province, China, under Grant 2015A030313204, in part by the Pearl River S&T Nova Program of Guangzhou under Grant 2014J2200052, in part by the National Natural Science Foundation of China under Grant 61203310, Grant 61503087 and Grant 61702239, in part by the Fundamental Research Funds for the Central Universities, SCUT, under Grant 2017MS043, in part by the Science Foundation of Jiangxi Provincial Department of Education under Grant GJJ170765 and Grant GJJ170798, and the Project of Jingdezhen Science and Technology Bureau under Grant 20161GYZD011-011.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ying, W., Zheng, K., Wu, Y., Li, J., Xu, X. (2020). Neural Architecture Search Using Multi-objective Evolutionary Algorithm Based on Decomposition. In: Li, K., Li, W., Wang, H., Liu, Y. (eds) Artificial Intelligence Algorithms and Applications. ISICA 2019. Communications in Computer and Information Science, vol 1205. Springer, Singapore. https://doi.org/10.1007/978-981-15-5577-0_11
Download citation
DOI: https://doi.org/10.1007/978-981-15-5577-0_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-5576-3
Online ISBN: 978-981-15-5577-0
eBook Packages: Computer ScienceComputer Science (R0)