Abstract
Neural architecture search (NAS) is a promising method to ascertain network architecture automatically and to build a suitable network for a particular application without any human intervention. However, NAS requires huge computation resources to find the optimal parameters of a network in the training phase of each search. Because a trade-off generally exists between model size and accuracy in deep learning models, the model size tends to increase in pursuit of higher accuracy. In applications with limited resources, such as edge AI, reducing the network weight might be more important than improving its accuracy. Alternatively, achieving high accuracy with maximum resources might be more important. The objective of this research is to find a model with sufficient accuracy with a limited number of weights and to reduce the search time. We improve the Differentiable Network Search (DARTS) algorithm, one of the fastest NAS methods, by adding another constraint to the loss function, which limits the number of network weights. We evaluate the proposed algorithm using three constraints. Compared to the conventional DARTS algorithm, the proposed algorithm reduces the search time by up to 40% when the model size range is set properly. It achieves comparable accuracy with that of DARTS.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Sig. Process. Mag. 29(6), 82–97 (2012). https://doi.org/10.1109/MSP.2012.2205597
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014)
Krizhevsky, A.: Learning multiple layers of features from tiny images. University of Toronto (2012)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS 2012, vol. 1. pp. 1097–1105. Curran Associates Inc., Red Hook (2012)
Liu, C., et al.: Progressive neural architecture search. CoRR abs/1712.00559 (2017). http://arxiv.org/abs/1712.00559
Liu, H., Simonyan, K., Yang, Y.: Darts: differentiable architecture search (2019)
Real, E., et al.: Large-scale evolution of image classifiers. CoRR abs/1703.01041 (2017). http://arxiv.org/abs/1703.01041
Tan, M., Chen, B., Pang, R., Vasudevan, V., Le, Q.V.: MnasNet: platform-aware neural architecture search for mobile. CoRR abs/1807.11626 (2018). http://arxiv.org/abs/1807.11626
Wu, B., et al.: FbNet: hardware-aware efficient convnet design via differentiable neural architecture search. CoRR abs/1812.03443 (2018). http://arxiv.org/abs/1812.03443
Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. CoRR abs/1611.01578 (2016). http://arxiv.org/abs/1611.01578
Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. CoRR abs/1707.07012 (2017). http://arxiv.org/abs/1707.07012
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yamada, F., Tsuji, S., Kawaguchi, H., Inoue, A., Sakai, Y. (2021). A High-Speed Neural Architecture Search Considering the Number of Weights. In: Edelkamp, S., Möller, R., Rueckert, E. (eds) KI 2021: Advances in Artificial Intelligence. KI 2021. Lecture Notes in Computer Science(), vol 12873. Springer, Cham. https://doi.org/10.1007/978-3-030-87626-5_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-87626-5_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87625-8
Online ISBN: 978-3-030-87626-5
eBook Packages: Computer ScienceComputer Science (R0)