Improving Efficient Neural Architecture Search Using Out-net
Authors: 
Cong Liu, Qinghai Miao, Min HuangAuthors Info & Claims
Cong Liu, Qinghai Miao, Min HuangAuthors Info & ClaimsPages 266 - 271
References
[1]
Zoph, B., and Le, Q. V., 2017. “Neural architecture search with reinforcement learning”. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net.
[2]
Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., and Dean, J., 2018. “Efficient neural architecture search via parameter sharing”. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsma¨ssan, Stockholm, Sweden, July 10-15, 2018, J. G. Dy and A. Krause, eds., Vol. 80 of Proceedings of Machine Learning Research, PMLR, pp. 4092–4101.
[3]
Elsken, T., Metzen, J. H., Hutter, F., 2019. “Neural architecture search: A survey.”. J. Mach. Learn. Res., 20(55), pp. 1–21.
[4]
Baker, B., Gupta, O., Naik, N., and Raskar, R., 2017. “Designing neural network architectures using reinforcement learning”. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net.
[5]
Real, E., Moore, S., Selle, A., Saxena, S., Suematsu, Y. L., Tan, J., Le, Q. V., and Kurakin, A., 2017. “Largescale evolution of image classifiers”. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, D. Precup and Y. W. Teh, eds., Vol. 70 of Proceedings of Machine Learning Research, PMLR, pp. 2902–2911.
[6]
Shu, Y., Wang, W., and Cai, S., 2020. “Understanding architectures learnt by cell-based neural architecture search”. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, OpenReview.net.
[7]
Baker, B., Gupta, O., Raskar, R., and Naik, N., 2017. “Accelerating neural architecture search using performance prediction”. arXiv preprint arXiv:1705.10823.
[8]
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Chen, X., and Wang, X., 2020. “A comprehensive survey of neural architecture search: Challenges and solutions”. arXiv preprint arXiv:2006.02903.
[9]
Zheng, X., Ji, R., Tang, L., Zhang, B., Liu, J., and Tian, Q., 2019. “Multinomial distribution learning for effective neural architecture search”. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, IEEE, pp. 1304–1313.
[10]
Elsken, T., Metzen, J.-H., and Hutter, F., 2017. “Simple and efficient architecture search for convolutional neural networks”. arXiv preprint arXiv:1711.04528.
[11]
Ghiasi, G., Lin, T., and Le, Q. V., 2019. “NAS-FPN: learning scalable feature pyramid architecture for object detection”. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, Computer Vision Foundation / IEEE, pp. 7036–7045.
[12]
Peng, J., Sun, M., Zhang, Z., Tan, T., and Yan, J., 2019. “Efficient neural architecture transformation search in channel-level for object detection”. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch´e-Buc, E. B. Fox, and R. Garnett, eds., pp. 14290–14299.
[13]
Liang, F., Lin, C., Guo, R., Sun, M., Wu, W., Yan, J., and Ouyang, W., 2020. “Computation reallocation for object detection”. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, OpenReview.net.
[14]
Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., and Sun, J., 2020. “Single path one-shot neural architecture search with uniform sampling”. In European Conference on Computer Vision, Springer, pp. 544–560.
[15]
Zhang, Y., Qiu, Z., Liu, J., Yao, T., Liu, D., and Mei, T., 2019. “Customizable architecture search for semantic segmentation”. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, Computer Vision Foundation / IEEE, pp. 11641–11650.
[16]
Liu, C., Chen, L., Schroff, F., Adam, H., Hua, W., Yuille, A. L., and Li, F., 2019. “Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation”. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, Computer Vision Foundation / IEEE, pp. 82–92.
[17]
Gong, X., Chang, S., Jiang, Y., and Wang, Z., 2019. “Autogan: Neural architecture search for generative adversarial networks”. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, IEEE, pp. 3223–3233.
[18]
Li, X., Zhou, Y., Pan, Z., and Feng, J., 2019. “Partial order pruning: For best speed/accuracy trade-off in neural architecture search”. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, Computer Vision Foundation / IEEE, pp. 9145–9153.
[19]
Fedorov, I., Adams, R. P., Mattina, M., and Whatmough, P. N., 2019. “Sparse: Sparse architecture search for cnns on resource-constrained microcontrollers”. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch´e-Buc, E. B. Fox, and R. Garnett, eds., pp. 4978–4990.
[20]
Elsken, T., Metzen, J. H., and Hutter, F., 2019. “Efficient multi-objective neural architecture search via lamarckian evolution”. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, OpenReview.net.
[21]
Zoph, B., and Le, Q. V., 2017. “Neural architecture search with reinforcement learning”. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net.
[22]
Yang, Z., Wang, Y., Chen, X., Shi, B., Xu, C., Xu, C., Tian, Q., and Xu, C., 2020. “CARS: continuous evolution for efficient neural architecture search”. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, IEEE, pp. 1826–1835.
[23]
Real, E., Aggarwal, A., Huang, Y., and Le, Q. V., 2019. “Regularized evolution for image classifier architecture search”. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, AAAI Press, pp. 4780–4789.
[24]
Liu, H., Simonyan, K., and Yang, Y., 2019. “DARTS: differentiable architecture search”. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, OpenReview.net.
[25]
He, K., Zhang, X., Ren, S., and Sun, J., 2016. “Deep residual learning for image recognition”. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, IEEE Computer Society, pp. 770–778.
[26]
Zoph, B., Vasudevan, V., Shlens, J., and Le, Q. V., 2018. “Learning transferable architectures for scalable image recognition”. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, IEEE Computer Society, pp. 8697–8710.
[27]
Hinton, G., Vinyals, O., and Dean, J., 2015. “Distilling the knowledge in a neural network”. stat, 1050, p. 9.
[28]
Williams, R. J., 1992. “Simple statistical gradient-following algorithms for connectionist reinforcement learning”. Machine learning, 8(3-4), pp. 229–256.
[29]
Kingma, D. P., and Ba, J., 2015. “Adam: A method for stochastic optimization”. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y. Bengio and Y. LeCun, eds.
[30]
Zhang, Miao, "Overcoming multi-model forgetting in one-shot nas with diversity maximization." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.
[31]
He, K., Zhang, X., Ren, S., & Sun, J. (2016, October). Identity mappings in deep residual networks. In European conference on computer vision (pp. 630-645). Springer, Cham.
Recommendations
A Training-free Genetic Neural Architecture Search
ACM ICEA '21: Proceedings of the 2021 ACM International Conference on Intelligent Computing and its Emerging ApplicationsThe so-called neural architecture search (NAS) provides an alternative way to construct a "good neural architecture," which would normally outperform hand-made architectures, for solving complex problems without domain knowledge. However, a critical ...
Auto-Keras: An Efficient Neural Architecture Search System
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data MiningNeural architecture search (NAS) has been proposed to automatically tune deep neural networks, but existing search algorithms, e.g., NASNet, PNAS, usually suffer from expensive computational cost. Network morphism, which keeps the functionality of a ...
Neural architecture search using progressive evolution
GECCO '22: Proceedings of the Genetic and Evolutionary Computation ConferenceVanilla neural architecture search using evolutionary algorithms (EA) involves evaluating each architecture by training it from scratch, which is extremely time-consuming. This can be reduced by using a supernet to estimate the fitness of every ...
Comments
Information & Contributors
Information
Published In
April 2021
498 pages
ISBN:9781450389501
DOI:10.1145/3467707
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 September 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCAI '21
ICCAI '21: 2021 7th International Conference on Computing and Artificial Intelligence
April 23 - 26, 2021
Tianjin, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 313Total Downloads
- Downloads (Last 12 months)106
- Downloads (Last 6 weeks)9
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in