A progressive genetic-based neural architecture search

Recently, more and more attention has been put forth on the application and deep learning, due to the widespread practicability of neural network computation. The purpose of this paper is developing an effective algorithm to automatically discover the optimal neural network architecture for several real applications.

The author proposes a novel algorithm, namely, progressive genetic-based neural architecture search (PG-NAS), as a solution to efficiently find the optimal neural network structure for given data. PG-NAS also employs several operations to effectively shrink the search space to reduce the computation cost and improve the accuracy validation.

The proposed PG-NAS could be utilized on several tasks for discovering the optimal network structure. The author reduces the demand of manual settings when implementing artificial intelligence (AI) models; hence, PG-NAS requires less human intervention than traditional machine learning. The average and top-1 metrics, such as error, loss and accuracy, are used to measure the discovered neural architectures of the proposed model over all baselines. The experimental results show that, with several real datasets, the proposed PG-NAS model consistently outperforms the state-of-the-art models in all metrics.

Generally, the size and the complexity of the search space for the neural network dominates the performance of computation time and resources. In this study, PG-NAS utilizes genetic operations to effectively generate the compact candidate set, i.e. fewer combinations need to be generated when constructing the candidate set. Moreover, by the proposed selector in PG-NAS, the non-promising network structure could be significantly pruned off. In addition, the accuracy derivation of each combination in the candidate set is also a performance bottleneck. The author develops a predictor network to efficiently estimate the accuracy to avoid the time-consuming derivation. The learning of the prediction process is also adjusted dynamically; this adaptive learning of the predictor could capture the pattern of training data effectively and efficiently. Furthermore, the proposed PG-NAS algorithm is applied on several real datasets to show its practicability and scalability.