research-article

Accelerating nonlinear DC circuit simulation with reinforcement learning

Authors:

Dan NiuAuthors Info & Claims

DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference

Pages 619 - 624

https://doi.org/10.1145/3489517.3530512

Published: 23 August 2022 Publication History

Abstract

DC analysis is the foundation for nonlinear electronic circuit simulation. Pseudo transient analysis (PTA) methods have gained great success among various continuation algorithms. However, PTA tends to be computationally intensive without careful tuning of parameters and proper stepping strategies. In this paper, we harness the latest advancing in machine learning to resolve these challenges simultaneously. Particularly, an active learning is leveraged to provide a fine initial solver environment, in which a TD3-based Reinforcement Learning (RL) is implemented to accelerate the simulation on the fly. The RL agent is strengthen with dual agents, priority sampling, and cooperative learning to enhance its robustness and convergence. The proposed algorithms are implemented in an out-of-the-box SPICElike simulator, which demonstrated a significant speedup: up to 3.1X for the initial stage and 234X for the RL stage.

References

[1]

J. Deng, K. Batselier, Y. Zhang, and N. Wong, "An efficient two-level dc operating points finder for transistor circuits," in DAC, pp. 1--6, 2014.

[2]

Z. Jin, T. Feng, Y. Duan, X. Wu, M. Cheng, Z. Zhou, and W. Liu, "PALBBD: A parallel arclength method using bordered block diagonal form for DC analysis," in GLSVLSI, pp. 327--332, 2021.

Digital Library

[3]

K. Kundert, The Designer's Guide to SPICE and SPECTRE®. Springer Science & Business Media, 2006.

[4]

T. Najibi, "Continuation methods as applied to circuit simulation," IEEE Circuits and Devices Magazine, pp. 48--49, 1989.

[5]

C. T. Kelley and D. E. Keyes, "Convergence analysis of pseudo-transient continuation," SIAM Journal on Numerical Analysis, pp. 508--523, 1998.

Digital Library

[6]

C. Lemke, "Pathways to solutions, fixed points, and equilibria (cb garcia and wj zangwill)," SIAM Review, pp. 445--446, 1984.

[7]

J. Zhou, L. Meiping, and W. Xiao, "An adaptive dynamic-element pta method for solving nonlinear dc operating point of transistor circuits," in MWSCAS, pp. 37--40, 2018.

[8]

X. Wu, Z. Jin, D. Niu, and Y. Inoue, "An adaptive time-step control method in damped pseudo-transient analysis for solving nonlinear DC circuit equations," IEICE Trans. Fundam. Electron. Commun. Comput. Sci., pp. 619--628, 2017.

[9]

W. W. Xing, X. Jin, Y. Liu, D. Niu, W. Zhao, and Z. Jin, "Boa-pta, a bayesian optimization accelerated error-free spice solver," arXiv preprint arXiv:2108.00257, 2021.

[10]

X. Wu, Z. Jin, D. Niu, and Y. Inoue, "A pta method using numerical integration algorithms with artificial damping for solving nonlinear dc circuits," Nonlinear Theory and Its Applications, IEICE, vol. 5, pp. 512--522, 2014.

[11]

S. Fujimoto, H. Hoof, and D. Meger, "Addressing function approximation error in actor-critic methods," in International Conference on Machine Learning, pp. 1587--1596, 2018.

[12]

C. Millán-Arias, B. J. T. Fernandes, F. Cruz, R. Dazeley, and S. Fernandes, "A robust approach for continuous interactive actor-critic algorithms," IEEE Access, pp. 104242--104260, 2021.

[13]

F. Rezazadeh, H. Chergui, L. Alonso, and C. V. Verikoukis, "Continuous multi-objective zero-touch network slicing via twin delayed DDPG and openai gym," CoRR, 2021.

[14]

C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. MIT Press, 2006.

Digital Library

[15]

S. Zhang, W. Lyu, F. Yang, C. Yan, D. Zhou, and X. Zeng, "Bayesian optimization approach for analog circuit synthesis using neural network," in DATE, pp. 1463--1468, 2019.

Cited By

Jiang JDuan YJin ZNakamura YWang Y(2025)Boosting the Performance of Transistor-Level Circuit Simulation with GNNProceedings of the 30th Asia and South Pacific Design Automation Conference10.1145/3658617.3703149(114-120)Online publication date: 20-Jan-2025
https://dl.acm.org/doi/10.1145/3658617.3703149
Dong YNiu DJin ZZhang CSun CZhou Z(2024)ISPT-Net: A Noval Transient Backward-Stepping Reduction Policy by Irregular Sequential Prediction Transformer2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546793(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546793
Jin ZFeng TWu XNiu DZhou ZZhuo C(2024)MSH: A Multi-Stage HiZ-Aware Homotopy Framework for Nonlinear DC Analysis2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546783(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546783
Show More Cited By

Index Terms

Accelerating nonlinear DC circuit simulation with reinforcement learning
1. Hardware
  1. Electronic design automation
    1. Methodologies for EDA
      1. Software tools for EDA

Recommendations

Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems

Recent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...
Reinforcement learning algorithms: A brief survey
Highlights
- RL can be used to solve problems involving sequential decision-making.
- RL is based on trial-and-error learning through rewards and punishments.
- The ultimate goal of an RL agent is to maximize cumulative reward.
- RL agent tries ...
Abstract
Reinforcement Learning (RL) is a machine learning (ML) technique to learn sequential decision-making in complex problems. RL is inspired by trial-and-error based human/animal learning. It can learn an optimal policy autonomously with knowledge ...
Relational Reinforcement Learning

Relational reinforcement learning is presented, a learning technique that combines reinforcement learning with relational learning or inductive logic programming. Due to the use of a more expressive representation language to represent states, actions ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference

July 2022

1462 pages

ISBN:9781450391429

DOI:10.1145/3489517

General Chair:
Rob Oshana
NXP

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGDA: ACM Special Interest Group on Design Automation
IEEE CEDA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 August 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

State Key Laboratory of Computer Architecture
Science Foundation of China University of Petroleum, Beijing

Conference

DAC '22

Sponsor:

SIGDA

DAC '22: 59th ACM/IEEE Design Automation Conference

July 10 - 14, 2022

California, San Francisco

Acceptance Rates

Overall Acceptance Rate 1,770 of 5,499 submissions, 32%

Upcoming Conference

DAC '25

Sponsor:
sigda

62nd ACM/IEEE Design Automation Conference

June 22 - 26, 2025

San Francisco , CA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
319
Total Downloads

Downloads (Last 12 months)83
Downloads (Last 6 weeks)12

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang JDuan YJin ZNakamura YWang Y(2025)Boosting the Performance of Transistor-Level Circuit Simulation with GNNProceedings of the 30th Asia and South Pacific Design Automation Conference10.1145/3658617.3703149(114-120)Online publication date: 20-Jan-2025
https://dl.acm.org/doi/10.1145/3658617.3703149
Dong YNiu DJin ZZhang CSun CZhou Z(2024)ISPT-Net: A Noval Transient Backward-Stepping Reduction Policy by Irregular Sequential Prediction Transformer2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546793(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546793
Jin ZFeng TWu XNiu DZhou ZZhuo C(2024)MSH: A Multi-Stage HiZ-Aware Homotopy Framework for Nonlinear DC Analysis2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546783(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546783
Chen PNiu DJin ZSun CLi QYan H(2024)TSA-TICER: A Two-Stage TICER Acceleration Framework for Model Order Reduction2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546520(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546520
Ouyang LJin CChen Q(2024)A Fast Recycling GMRES Method With Smart Frequency Sweeping for Efficient Periodic Small-Signal AnalysisIEEE Transactions on Circuits and Systems II: Express Briefs10.1109/TCSII.2024.337240471:8(3765-3769)Online publication date: Aug-2024
https://doi.org/10.1109/TCSII.2024.3372404
Jin ZLi WBai YWang TLu YLiu WKim T(2024)Machine Learning and GPU Accelerated Sparse Linear Solvers for Transistor-Level Circuit Simulation: A Perspective Survey (Invited Paper)Proceedings of the 29th Asia and South Pacific Design Automation Conference10.1109/ASP-DAC58780.2024.10473846(96-101)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1109/ASP-DAC58780.2024.10473846
Niu DDong YJin ZZhang CLi QSun C(2023)OSSP-PTA: An Online Stochastic Stepping Policy for PTA on Reinforcement LearningIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2023.325173142:11(4310-4323)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1109/TCAD.2023.3251731
Xu LNiu DYang YWang AJin ZDong YZhang C(2023)Adaptive Transient Stepping Policy on Reinforcement Learning2023 International Symposium of Electronics Design Automation (ISEDA)10.1109/ISEDA59274.2023.10218535(46-51)Online publication date: 8-May-2023
https://doi.org/10.1109/ISEDA59274.2023.10218535

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten