skip to main content
research-article

ESPSim: An Efficient Scalable Power Grid Simulator Based on Parallel Algebraic Multigrid

Authors Info & Claims
Published:10 December 2022Publication History
Skip Abstract Section

Abstract

Fast verification for the extremely large-scale power grid is demanding as CMOS technology advances consistently. In this work, we propose ESPSim, an efficient scalable power grid simulator based on a parallel smoothed aggregation-based algebraic multigrid technique. ESPSim has the ability to do fast DC and transient analysis through MPI and adaptive timestep control mechanism. Thanks to the smoother applied on the prolongation operator, ESPSim copes well with the convergence rate on extremely large-scale power grid transient analysis. Extensive experiments are conducted with a variety of serial/parallel solvers. The runtime of ESPSim is linear with case size. With 16 processors, 1,000 timesteps transient analysis of 63.4M nodes can be completed in 22.1 min. Over 22× speedup compared to the well-known direct solver Cholmod is observed.

REFERENCES

  1. [1] Check Memory. 2021. Checking Memory from a Parallel C Program. Retrieved from https://hpcf.umbc.edu/general-productivity/checking-memory-usage/.Google ScholarGoogle Scholar
  2. [2] Hypre User Guide. 2021. Hypre Software and Documentation. Retrieved from https://computing.llnl.gov/projects/hypre-scalable-linear-solvers-multigrid-methods.Google ScholarGoogle Scholar
  3. [3] xxHash User Guide. 2021. xxHash. Retrieved from https://github.com/Cyan4973/xxHash.Google ScholarGoogle Scholar
  4. [4] Parmetis User Guide. 2022. Parmetis Manual Version 4.0. Retrieved from http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview.Google ScholarGoogle Scholar
  5. [5] Achar Ramachandra, Nakhla Michel S., Dhindsa Harjot S., Sridhar Arvind R., Paul Douglas, and Nakhla Natalie M.. 2011. Parallel and scalable transient simulator for power grids via waveform relaxation (PTS-PWR). IEEE Trans. Very Large Scale Integr. Syst. 19, 2 (2011), 319332. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Adelmann Andreas, Arbenz Peter, and Ineichen Yves. 2010. Improvements of a fast parallel poisson solver on irregular domains. In Proceedings of the 10th International Conference on Applied Parallel and Scientific Computing (PARA’10)(Lecture Notes in Computer Science, Vol. 7133), Jónasson Kristján (Ed.). Springer, 6574. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Balay Satish, Abhyankar Shrirang, Adams Mark F., Benson Steven, Brown Jed, Brune Peter, Buschelman Kris, Constantinescu Emil M., Dalcin Lisandro, Dener Alp, Eijkhout Victor, Gropp William D., Hapla Václav, Isaac Tobin, Jolivet Pierre, Karpeev Dmitry, Kaushik Dinesh, Knepley Matthew G., Kong Fande, Kruger Scott, May Dave A., McInnes Lois Curfman, Mills Richard Tran, Mitchell Lawrence, Munson Todd, Roman Jose E., Rupp Karl, Sanan Patrick, Sarich Jason, Smith Barry F., Zampini Stefano, Zhang Hong, Zhang Hong, and Zhang Junchao. 2021. PETSc Web Page. Retrieved from https://petsc.org/.Google ScholarGoogle Scholar
  8. [8] Brezina Marian, Falgout Robert D., MacLachlan Scott, Manteuffel Thomas A., McCormick Stephen F., and Ruge John W.. 2004. Adaptive smoothed aggregation (\(\alpha\)SA). SIAM J. Sci. Comput. 25, 6 (2004), 18961920. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Briggs William L., Henson Van Emden, and McCormick Stephen F.. 2000. A Multigrid Tutorial, 2nd ed. SIAM.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Cai Yici, Shi Jin, Pan Zhu, Hong Xianlong, and Tan Sheldon X.-D.. 2008. Large scale P/G grid transient simulation using hierarchical relaxed approach. Integration 41, 1 (2008), 153160. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Chen Tsung-Hao and Chen Charlie Chung-Ping. 2001. Efficient large-scale power grid analysis based on preconditioned Krylov-subspace iterative methods. In Proceedings of the 38th Design Automation Conference. 559562. Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Chen Yanqing, Davis Timothy A., Hager William W., and Rajamanickam Sivasankaran. 2008. Algorithm 887: CHOLMOD, supernodal sparse cholesky factorization and update/downdate. ACM Trans. Math. Softw. 35, 3 (2008), 22:1–22:14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Cui Ganqu, Yu Wenjian, Li Xin, Zeng Zhiyu, and Gu Ben. 2019. Machine-learning-driven matrix ordering for power grid analysis. In Proceedings of the Design, Automation Test in Europe Conference Exhibition (DATE’19). 984987. Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Davis Timothy A.. [n.d.]. SuiteSparse. Retrieved from https://people.engr.tamu.edu/davis/suitesparse.html.Google ScholarGoogle Scholar
  15. [15] Sterck Hans De, Falgout Robert D., Nolting Joshua W., and Yang Ulrike Meier. 2008. Distance-two interpolation for parallel algebraic multigrid. Numer. Linear Algebra Appl. 15, 2–3 (2008), 115139. Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Sterck Hans De, Yang Ulrike Meier, and Heys Jeffrey J.. 2006. Reducing complexity in parallel algebraic multigrid preconditioners. SIAM J. Matrix Anal. Appl. 27, 4 (2006), 10191039. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Demidov D.. 2019. AMGCL: An efficient, flexible, and extensible algebraic multigrid implementation. Lobachevskii J. Math. 40, 5 (May2019), 535546. Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Demmel James. 1997. Applied Numerical Linear Algebra. SIAM. Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Feng Zhuo and Li Peng. 2008. Multigrid on GPU: Tackling power grid analysis on parallel SIMT platforms. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. 647654. Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Grigori Laura, Demmel James, and Li Xiaoye S.. 2007. Parallel symbolic factorization for sparse LU with static pivoting. SIAM J. Sci. Comput. 29, 3 (2007), 12891314. Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Hao Zhuang. 2016. Exponential Time Integration for Transient Analysis of Large-Scale Circuits. Ph.D. Dissertation. Department of Computer Science, University of California, San Diego, CA. Retrieved from https://escholarship.org/uc/item/60d8c2r6.Google ScholarGoogle Scholar
  22. [22] He Qing, Au William, Korobkov Alexander, and Venkateswaran Subramanian. 2014. Parallel power grid analysis using distributed direct linear solver. In Proceedings of the IEEE International Symposium on Electromagnetic Compatibility (EMC’14). 866871. Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Hestenes Magnus R., Stiefel Eduard, et al. 1952. Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bureau Stand. 49, 6 (1952), 409436.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Kielkowski R. M.. 1998. Inside SPICE. McGraw-Hill.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Le Zhang. 2015. Parallel simulation for VLSI power grid. Ph.D. Dissertation. the Department of Computer Science & Engineering, Texas A&M University.Google ScholarGoogle Scholar
  26. [26] Lee Yu-Min and Chen C. C.-P.. 2002. Power grid transient simulation in linear time based on transmission-line-modeling alternating-direction-implicit method. IEEE Trans. Comput.-Aided Design Integr. Circ. Syst. 21, 11 (2002), 13431352. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Lee Yu-Min and Ho Chia-Tung. 2017. InTraSim: Incremental transient simulation of power grids. IEEE Trans. Comput.-Aided Design Integr. Circ. Syst. 36, 12 (2017), 20522065. Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Li P.. 2021. IBM Power Grid Benchmarks. Retrieved from https://web.ece.ucsb.edu/lip/PGBenchmarks/ibmpgbench.html.Google ScholarGoogle Scholar
  29. [29] Lin Calvin and Snyder Lawrence. 2008. Principles of Parallel Programming. Pearson.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Liu Zhiqiang, Yu Wenjian, and Feng Zhuo. 2021. feGRASS: Fast and effective graph spectral sparsification for scalable power grid analysis. IEEE Trans. Comput.-Aided Design Integr. Circ. Syst. (2021), 11. Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Nassif S. R. and Kozhaya J. N.. 2000. Fast power grid simulation. In Proceedings of the 37th Design Automation Conference. 156161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Notay Y.. 2010. An aggregation-based algebraic multigrid method. Electr. Trans. Numer. Anal. 37 (2010), 123146.Google ScholarGoogle Scholar
  33. [33] Notay Yvan. 2021. AGMG Software and Documentation Documentation. Retrieved from http://homepages.ulb.ac.be/ynotay/AGMG.Google ScholarGoogle Scholar
  34. [34] Notay Yvan and Napov Artem. 2015. A massively parallel solver for discrete Poisson-like problems. J. Comput. Phys. 281 (2015), 237250. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Pan Zhu, Cai Yici, Tan S. X.-D., Luo Zuying, and Hong Xianlong. 2004. Transient analysis of on-chip power distribution networks using equivalent circuit modeling. In Proceedings of the International Symposium on Signals, Circuits and Systems (SCS’03). 6368. Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Qian Haifeng, Nassif S. R., and Sapatnekar S. S.. 2005. Power grid analysis using random walks. IEEE Trans. Comput.-Aided Design Integr. Circ. Syst. 24, 8 (2005), 12041224. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Saad Yousef. 2003. Iterative Methods for Sparse Linear Systems. SIAM.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Sedgewick Robert and Wayne Kevin. 2014. Algorithms, Part I, 4th ed. Pearson Education.Google ScholarGoogle Scholar
  39. [39] Sun Kai, Zhou Quming, Mohanram Kartik, and Sorensen Danny C.. 2007. Parallel domain decomposition for simulation of large-scale power grids. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. 5459. Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Vanek Petr, Brezina Marian, and Mandel Jan. 2001. Convergence of algebraic multigrid based on smoothed aggregation. Numer. Math. 88, 3 (2001), 559579. Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Vaněk P., Mandel J., and Brezina M.. 1996. Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems. Computing 56, 3 (Sep.1996), 179196. Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Wang Jia and Xiong Xuanxing. 2013. Scalable power grid transient analysis via MOR-assisted time-domain simulations. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD’13). 548552. Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Webster R.. 2001. Performance of algebraic multi-grid solvers based on unsmoothed and smoothed aggregation schemes. Int. J. Numer. Methods Fluids 36, 7 (2001), 743772.Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Yang J. and Li Z.. 2021. THU Power Grid Benchmarks. Retrieved from http://tiger.cs.tsinghua.edu.cn/PGBench/.Google ScholarGoogle Scholar
  45. [45] Yang Jianlei, Li Zuowei, Cai Yici, and Zhou Qiang. 2012. PowerRush: Efficient transient simulation for power grid analysis. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD’12). 653659.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Yang Jianlei, Li Zuowei, Cai Yici, and Zhou Qiang. 2014. PowerRush: An efficient simulator for static power grid analysis. IEEE Trans. Very Large Scale Integr. Syst. 22, 10 (2014), 21032116. Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Yu Ting and Wong Martin. D. F.. 2012. PGT_SOLVER: An efficient solver for power grid transient analysis. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD’12). 647652.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Yu Ting, Xiao Zigang, and Wong Martin D. F.. 2012. Efficient parallel power grid analysis via additive Schwarz method. In Proceedings of the International Conference on Computer-Aided Design (ICCAD’12). ACM, New York, NY, 399406. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Zhang Le and Sarin Vivek. 2016. Parallel power grid analysis based on enlarged partitions. ACM Trans. Design Autom. Electr. Syst. 21, 2 (2016), 26:1–26:21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Zhiqiang Liu and Wenjian Yu. 2021. pGRASS-Solver: A parallel iterative solver for scalable power grid analysis based on graph spectral sparsification. Retrieved from https://numbda.cs.tsinghua.edu.cn/papers/iccad21_2.pdf.Google ScholarGoogle Scholar
  51. [51] Zhu Zhengyong, Yao Bo, and Cheng Chug-Kuan. 2003. Power network analysis using an adaptive algebraic multigrid approach. In Proceedings of the Design Automation Conference (DAC’03). 105108. Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Zhuang Hao, Weng Shih-Hung, Lin Jeng-Hau, and Cheng Chung-Kuan. 2014. MATEX: A distributed framework for transient simulation of power distribution networks. In Proceedings of the 51st Annual Design Automation Conference (DAC’14). ACM, New York, NY, 16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Zhuang Hao, Yu Wenjian, Kang Ilgweon, Wang Xinan, and Cheng Chung-Kuan. 2015. An algorithmic framework for efficient large-scale circuit simulation using exponential integrators. In Proceedings of the 52nd ACM/EDAC/IEEE Design Automation Conference (DAC’15). 16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. [54] Zhuo Cheng, Hu Jiang, Zhao Min, and Chen Kangsheng. 2008. Power grid analysis and optimization using algebraic multigrid. IEEE Trans. Comput.-Aided Design Integr. Circ. Syst. 27, 4 (2008), 738751. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. ESPSim: An Efficient Scalable Power Grid Simulator Based on Parallel Algebraic Multigrid

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Design Automation of Electronic Systems
            ACM Transactions on Design Automation of Electronic Systems  Volume 28, Issue 1
            January 2023
            321 pages
            ISSN:1084-4309
            EISSN:1557-7309
            DOI:10.1145/3573313
            Issue’s Table of Contents

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 10 December 2022
            • Online AM: 19 May 2022
            • Accepted: 29 March 2022
            • Revised: 21 February 2022
            • Received: 11 September 2021
            Published in todaes Volume 28, Issue 1

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Full Text

          View this article in Full Text.

          View Full Text

          HTML Format

          View this article in HTML Format .

          View HTML Format