Loading [a11y]/accessibility-menu.js
Surrogate-Assisted Evolutionary Multiobjective Neural Architecture Search Based on Transfer Stacking and Knowledge Distillation | IEEE Journals & Magazine | IEEE Xplore

Surrogate-Assisted Evolutionary Multiobjective Neural Architecture Search Based on Transfer Stacking and Knowledge Distillation


Abstract:

Multiobjective neural architecture search (MONAS) methods based on evolutionary algorithms (EAs) are inefficient when the evaluation of each architecture incorporates par...Show More

Abstract:

Multiobjective neural architecture search (MONAS) methods based on evolutionary algorithms (EAs) are inefficient when the evaluation of each architecture incorporates parameter learning from scratch. A surrogate-assisted multiobjective neural architecture search (MONAS) problem can be tough considering cold-start in surrogate construction, and the evaluation of predicted promising architectures could still be cumbersome. Previously solved MONAS problems are likely to convey useful knowledge that could assist solving the current MONAS problem. To take the benefit from knowledge of these previous practices, a framework tackling large-scale knowledge transfer is proposed. Through sparse-constraint transfer stacking, the surrogate for the current problem gets informative easily. With knee-region knowledge distillation from previously learned parameters of nondominated architectures, evaluation of current architectures could be efficient and credible. To avoid transferring knowledge from irrelevant problems, an iterative source selection algorithm is designed to avoid negative transfer. The proposed framework is analyzed under different source and target MONAS problem combinations. Results show that with the help of this framework, architectures with competitive performance could be found under limited evaluation budget.
Published in: IEEE Transactions on Evolutionary Computation ( Volume: 28, Issue: 3, June 2024)
Page(s): 608 - 622
Date of Publication: 26 September 2023

ISSN Information:

Funding Agency:


References

References is not available for this document.