Zhou Jin
ABSTRACT
DC analysis is the foundation for nonlinear electronic circuit simulation. Pseudo transient analysis (PTA) methods have gained great success among various continuation algorithms. However, PTA tends to be computationally intensive without careful tuning of parameters and proper stepping strategies. In this paper, we harness the latest advancing in machine learning to resolve these challenges simultaneously. Particularly, an active learning is leveraged to provide a fine initial solver environment, in which a TD3-based Reinforcement Learning (RL) is implemented to accelerate the simulation on the fly. The RL agent is strengthen with dual agents, priority sampling, and cooperative learning to enhance its robustness and convergence. The proposed algorithms are implemented in an out-of-the-box SPICElike simulator, which demonstrated a significant speedup: up to 3.1X for the initial stage and 234X for the RL stage.
- J. Deng, K. Batselier, Y. Zhang, and N. Wong, "An efficient two-level dc operating points finder for transistor circuits," in DAC, pp. 1--6, 2014.Google Scholar
- Z. Jin, T. Feng, Y. Duan, X. Wu, M. Cheng, Z. Zhou, and W. Liu, "PALBBD: A parallel arclength method using bordered block diagonal form for DC analysis," in GLSVLSI, pp. 327--332, 2021.Google ScholarDigital Library
- K. Kundert, The Designer's Guide to SPICE and SPECTRE®. Springer Science & Business Media, 2006.Google Scholar
- T. Najibi, "Continuation methods as applied to circuit simulation," IEEE Circuits and Devices Magazine, pp. 48--49, 1989.Google Scholar
- C. T. Kelley and D. E. Keyes, "Convergence analysis of pseudo-transient continuation," SIAM Journal on Numerical Analysis, pp. 508--523, 1998.Google ScholarDigital Library
- C. Lemke, "Pathways to solutions, fixed points, and equilibria (cb garcia and wj zangwill)," SIAM Review, pp. 445--446, 1984.Google ScholarCross Ref
- J. Zhou, L. Meiping, and W. Xiao, "An adaptive dynamic-element pta method for solving nonlinear dc operating point of transistor circuits," in MWSCAS, pp. 37--40, 2018.Google Scholar
- X. Wu, Z. Jin, D. Niu, and Y. Inoue, "An adaptive time-step control method in damped pseudo-transient analysis for solving nonlinear DC circuit equations," IEICE Trans. Fundam. Electron. Commun. Comput. Sci., pp. 619--628, 2017.Google ScholarCross Ref
- W. W. Xing, X. Jin, Y. Liu, D. Niu, W. Zhao, and Z. Jin, "Boa-pta, a bayesian optimization accelerated error-free spice solver," arXiv preprint arXiv:2108.00257, 2021.Google Scholar
- X. Wu, Z. Jin, D. Niu, and Y. Inoue, "A pta method using numerical integration algorithms with artificial damping for solving nonlinear dc circuits," Nonlinear Theory and Its Applications, IEICE, vol. 5, pp. 512--522, 2014.Google ScholarCross Ref
- S. Fujimoto, H. Hoof, and D. Meger, "Addressing function approximation error in actor-critic methods," in International Conference on Machine Learning, pp. 1587--1596, 2018.Google Scholar
- C. Millán-Arias, B. J. T. Fernandes, F. Cruz, R. Dazeley, and S. Fernandes, "A robust approach for continuous interactive actor-critic algorithms," IEEE Access, pp. 104242--104260, 2021.Google ScholarCross Ref
- F. Rezazadeh, H. Chergui, L. Alonso, and C. V. Verikoukis, "Continuous multi-objective zero-touch network slicing via twin delayed DDPG and openai gym," CoRR, 2021.Google Scholar
- C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. MIT Press, 2006.Google ScholarDigital Library
- S. Zhang, W. Lyu, F. Yang, C. Yan, D. Zhou, and X. Zeng, "Bayesian optimization approach for analog circuit synthesis using neural network," in DATE, pp. 1463--1468, 2019.Google Scholar
Index Terms
- Accelerating nonlinear DC circuit simulation with reinforcement learning
Recommendations
Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent SystemsRecent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...
Relational Reinforcement Learning
Relational reinforcement learning is presented, a learning technique that combines reinforcement learning with relational learning or inductive logic programming. Due to the use of a more expressive representation language to represent states, actions ...
Reinforcement learning algorithms: A brief survey
Highlights- RL can be used to solve problems involving sequential decision-making.
- RL is based on trial-and-error learning through rewards and punishments.
- The ultimate goal of an RL agent is to maximize cumulative reward.
- RL agent tries ...
AbstractReinforcement Learning (RL) is a machine learning (ML) technique to learn sequential decision-making in complex problems. RL is inspired by trial-and-error based human/animal learning. It can learn an optimal policy autonomously with knowledge ...
Comments