Yuejiang Wen
ABSTRACT
Bayesian optimization (BO) samples points of interest to update a surrogate model for a blackbox function. This makes it a powerful technique to optimize electronic designs which have unknown objective functions and demand high computational cost of simulation. Unfortunately, Bayesian optimization suffers from scalability issues, e.g., it can perform well in problems up to 20 dimensions. This paper addresses the curse of dimensionality and proposes an algorithm entitled Inspection-based Combo Random Embedding Bayesian Optimization (IC-REMBO). IC-REMBO improves the effectiveness and efficiency of the Random EMbedding Bayesian Optimization (REMBO) approach, which is a state-of-the-art high dimensional optimization method. Generally, it inspects the space near local optima to explore more points near local optima, so that it mitigates the over-exploration on boundaries and embedding distortion in REMBO. Consequently, it helps escape from local optima and provides a family of feasible solutions when inspecting near global optimum within a limited number of iterations.
The effectiveness and efficiency of the proposed algorithm are compared with the state-of-the-art REMBO when optimizing a mmWave receiver with 38 calibration parameters to meet 4 objectives. The optimization results are close to that of a human expert. To the best of our knowledge, this is the first time applying REMBO or inspection method to electronic design.
- Y. Wang, P. D. Franzon, D. Smart, and B. Swahn, "Multi-fidelity surrogate-based optimization for electromagnetic simulation acceleration," ACM Transactions on Design Automation of Electronic Systems (TODAES), vol. 25, no. 5, pp. 1--21, 2020.Google Scholar
Digital Library
- H. M. Torun, M. Swaminathan, A. K. Davis, and M. L. F. Bellaredj, "A global bayesian optimization algorithm and its application to integrated system design," IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 26, no. 4, pp. 792--802, 2018.Google ScholarCross Ref
- W. Lyu, F. Yang, C. Yan, D. Zhou, and X. Zeng, "Batch bayesian optimization via multi-objective acquisition ensemble for automated analog circuit design," in International Conference on machine learning. PMLR, 2018, pp. 3306--3314.Google Scholar
- J. Bergstra, D. Yamins, D. D. Cox et al., "Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms," in Proceedings of the 12th Python in science conference, vol. 13. Citeseer, 2013, p. 20.Google Scholar
- J. Snoek, H. Larochelle, and R. P. Adams, "Practical bayesian optimization of machine learning algorithms," Advances in neural information processing systems, vol. 25, 2012.Google Scholar
- E. Brochu, T. Brochu, and N. De Freitas, "A Bayesian interactive optimization approach to procedural animation design," in Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 2010, pp. 103--112.Google Scholar
- Z. Wang, M. Zoghi, F. Hutter, D. Matheson, and N. De Freitas, "Bayesian optimization in high dimensions via random embeddings," in Twenty-Third international joint conference on artificial intelligence, 2013.Google Scholar
- J. Djolonga, A. Krause, and V. Cevher, "High-dimensional gaussian process bandits," in Neural Information Processing Systems, no. CONF, 2013.Google Scholar
- C.-L. Li, K. Kandasamy, B. Poczos, and J. Schneider, "High dimensional bayesian optimization via restricted projection pursuit models," in Artificial Intelligence and Statistics. PMLR, 2016, pp. 884--892.Google Scholar
- A. Nayebi, A. Munteanu, and M. Poloczek, "A framework for Bayesian optimization in embedded subspaces," in International Conference on Machine Learning. PMLR, 2019, pp. 4752--4761.Google Scholar
- R. Moriconi, K. S. Kumar, and M. P. Deisenroth, "High-dimensional bayesian optimization with manifold gaussian processes," 2019.Google Scholar
- P.-I. Schneider, X. G. Santiago, C. Rockstuhl, and S. Burger, "Global optimization of complex optical structures using bayesian optimization based on gaussian processes," in Digital Optical Technologies 2017, vol. 10335. International Society for Optics and Photonics, 2017, p. 103350O.Google Scholar
- H. Qian, Y.-Q. Hu, and Y. Yu, "Derivative-free optimization of high-dimensional non-convex functions by sequential random embeddings." in IJCAI, 2016, pp. 1946--1952.Google Scholar
- J. Quinonero-Candela and C. E. Rasmussen, "A unifying view of sparse approximate gaussian process regression," The Journal of Machine Learning Research, vol. 6, pp. 1939--1959, 2005.Google ScholarDigital Library
- M. Zhang, H. Li, and S. Su, "High dimensional bayesian optimization via supervised dimension reduction," arXiv preprint arXiv:1907.08953, 2019.Google Scholar
- R. Moriconi, M. P. Deisenroth, and K. S. Kumar, "High-dimensional bayesian optimization using low-dimensional feature spaces," Machine Learning, vol. 109, no. 9, pp. 1925--1943, 2020.Google ScholarDigital Library
- C. Li, S. Gupta, S. Rana, V. Nguyen, S. Venkatesh, and A. Shilton, "High dimensional bayesian optimization using dropout," arXiv preprint arXiv:1802.05400, 2018.Google Scholar
- K. Touloupas, N. Chouridis, and P. P. Sotiriadis, "Local bayesian optimization for analog circuit sizing," in 2021 58th ACM/IEEE Design Automation Conference (DAC). IEEE, 2021, pp. 1237--1242.Google Scholar
- H. M. Torun and M. Swaminathan, "High-dimensional global optimization method for high-frequency electronic design," IEEE Transactions on Microwave Theory and Techniques, vol. 67, no. 6, pp. 2128--2142, 2019.Google ScholarCross Ref
- M. Malu, G. Dasarathy, and A. Spanias, "Bayesian optimization in high-dimensional spaces: A brief survey," in 2021 12th International Conference on Information, Intelligence, Systems & Applications (IISA). IEEE, 2021, pp. 1--8.Google Scholar
- Y. Chen, Y. Sun, and W. Yin, "Run-and-inspect method for nonconvex optimization and global optimality bounds for r-local minimizers," Mathematical Programming, vol. 176, no. 1, pp. 39--67, 2019.Google ScholarDigital Library
- F. Kashfi, S. Hatami, and M. Pedram, "Multi-objective optimization techniques for vlsi circuits," in 2011 12th International Symposium on Quality Electronic Design. IEEE, 2011, pp. 1--8.Google Scholar
- J. Dean, S. Hari, A. Bhat, and B. A. Floyd, "A 4--31ghz direct-conversion receiver employing frequency-translated feedback," in ESSCIRC 2021-IEEE 47th European Solid State Circuits Conference (ESSCIRC). IEEE, 2021, pp. 187--190.Google Scholar
Index Terms
- High Dimensional Optimization for Electronic Design
Recommendations
High Dimensional Bayesian Optimization with Kernel Principal Component Analysis
Parallel Problem Solving from Nature – PPSN XVIIAbstractBayesian Optimization (BO) is a surrogate-based global optimization strategy that relies on a Gaussian Process regression (GPR) model to approximate the objective function and an acquisition function to suggest candidate points. It is well-known ...
Micro-particle swarm optimizer for solving high dimensional optimization problems (µPSO for high dimensional optimization problems)
A simple and powerful micro-particle swarm optimizer (@mPSO) is presented. For high dimensional optimization problems, the proposed @mPSO needs only a small population to outperform the standard particle swarm optimizer (PSO) that uses a larger ...
A novel neural-network approach of analog fault diagnosis based on kernel discriminant analysis and particle swarm optimization
Kernel principal component analysis (KPCA) and kernel linear discriminant analysis (KLDA) are two commonly used and effective methods for dimensionality reduction and feature extraction. In this paper, we propose a KLDA method based on maximal class ...
Comments