Ismail S. Bustany
Jinwook Jung
Patrick H. Madden
ABSTRACT
Circuit benchmarks for VLSI physical design have been growing in size and complexity, helping the industry tackle new problems and find new approaches. In this paper, we take a look back at how benchmarking efforts have shaped the research community, consider trade-offs that have been made, and speculate on what may come next.
- G. E. Moore. Cramming more components onto integrated circuits. Electronics Magazine, 38(8):114--117, April 1965.Google Scholar
- C. Mead and L. Conway. Introduction to VLSI Systems. Addison-Wesley, 1993.Google Scholar
- B. Preas. Benchmarks for cell-based layout systems. In Proc. Design Automation Conf, pages 319 -- 320, 1987.Google ScholarDigital Library
- K. Kozminski. Benchmarks for layout synthesis - evolution and current status. In Proc. Design Automation Conf, pages 265 -- 270, 1991.Google Scholar
- S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220(4598):671--680, May 1983.Google ScholarCross Ref
- J. Kleinhans, G. Sigl, F. Johannes, and K. Antreich. GORDIAN: VLSI placement by quadratic programming and slicing optimization. IEEE Trans. on Computer-Aided Design of Integrated Circuits and Systems, 10(3):356--365, 1991.Google ScholarDigital Library
- A. E. Dunlop and B. W. Kernighan. A procedure for placement of standard-cell VLSI circuits. IEEE Trans. on Computer-Aided Design of Integrated Circuits and Systems, CAD-4(1):92--98, January 1985.Google ScholarDigital Library
- P. H. Madden. Reporting of standard cell placement results. In Proc. Int. Symp. on Physical Design, pages 30--35, 2001.Google ScholarDigital Library
- C. J. Alpert. The ISPD98 circuit benchmark suite. In Proc. Int. Symp. on Physical Design, pages 80--85, 1998.Google Scholar
- M. Wang, X. Yang, and M. Sarrafzadeh. Dragon2000: Standard-cell placement tool for large industry circuits. In Proc. Int. Conf. on Computer Aided Design, pages 260--263, 2000.Google ScholarDigital Library
- S. N. Adya and I. L. Markov. Consistent placement of macroblock using floorplanning and standard-cell placement. In Proc. Int. Symp. on Physical Design, pages 12--17, 2002.Google Scholar
- A. E. Caldwell, Y. Cao, A. B. Kahng, F. Koushanfar, H. Lu, I. L. Markov, M. Oliver, D. Stroobandt, and D. Sylvester. GTX: The MARCO GSRC technology extrapolation system. In Proc. Design Automation Conf, pages 693--698, 2000.Google ScholarDigital Library
- GSRC. The GSRC bookshelf. http://www.gigascale.org/bookshelf.Google Scholar
- G.-J. Nam, C. J. Alpert, P. Villarrubia, B. Winter, and M. Yildiz. The ISPD2005 placement contest and benchmark suite. In Proc. Int. Symp. on Physical Design, pages 216--220, 2005.Google Scholar
- V. Yutsis, I. S. Bustany, D. Chinnery, J. R. Shinnerl, and W.-H. Liu. ISPD 2014 benchmarks with sub-45nm technology rules for detailed-routing-driven placement. In Proc. ISPD, pages 161--168, 2014.Google Scholar
- I. S. Bustany, D. Chinnery, J. R. Shinnerl, and V. Yutsis. ISPD 2015 benhmarks with fence regions and routing blockages for detailed-routing-driven placement. In Proc. ISPD, pages 157--164, 2015.Google Scholar
- N. K. Darav, A. Kennings, D. Westwick, and L. Behjat. High performance global placement legalization accounting for fence regions. In ICCAD, pages 514--519, 2015.Google ScholarDigital Library
- N. K. Darav, A. Kennings, A. Fakheri Tabrizi, D. Westwick, and L. Behjat. Eh?Placer: A high-performance modern technology-driven placer. ACM Trans. Design Automation of Electronic Systems, 21, July 2016.Google ScholarDigital Library
- C. Huang, C.-H. Chiou, K.-H. Tseng, and Y. Chang. Detailed-routing-driven analytical standard-cell placement. In PRoc. ASPDAC, pages 378--383, 2015.Google ScholarCross Ref
- V. Betz and J. Rose. VPR: A new packing, placement and routing tool for FPGA research. In Proceedings of the 7th International Workshop on Field-Programmable Logic and Applications, pages 213--222, 1997.Google ScholarDigital Library
- Stephen Yang, Aman Gayasen, Chandra Mulpuri, Sainath Reddy, and Rajat Aggarwal. Routability driven FPGA placement contest. In International Symposium on Physical Design (ISPD), pages 139--143, 2016.Google ScholarDigital Library
- Wuxi Li, Shounak Dhar, and David Z. Pan. UTPlaceF: A routability-driven fpga placer with physical and congestion aware packing. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), 2017.Google Scholar
- C.-W. Pui, G. Chen, W.-K. Chow, K.-C. Lam, J. Kuang, P. Tu, H. Zhang, E. F. Y. Young, and B. Yu. RippleFPGA: A routabilitydriven placement for large-scale heterogeneous FPGAs. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), page 67:1--67:8, 2016.Google ScholarDigital Library
- R. Pattison, Z. Abuowaimer, S. Areibi, G. Grewal, and A. Vannelli. GPlace: A congestion-aware placement tool for ultrascale FPGAs. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), page 68:1--68:7, 2016.Google ScholarDigital Library
- Stephen Yang, Chandra Mulpuri, Sainath Reddy, Meghraj Kalase, Srinivasan Dasasathyan, Mehrdad E. Dehkordi, Marvin Tom, and Rajat Aggarwal. Clock aware FPGA placement contest. In International Symposium on Physical Design (ISPD), pages 159--164, 2017.Google ScholarDigital Library
- Wuxi Li, Yibo Lin, Meng Li, Shounak Dhar, and David Z. Pan. UTPlaceF 2.0: A high-performance clock-aware FPGA placement engine. ACM Transactions on Design Automation of Electronic Systems (TODAES), December 2017.Google Scholar
- Y.-C. Chen, S.-Y. Chen, and Y.-W. Chang. Efficient and effective packing and analytical placement for large-scale heterogeneous FPGAs. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), page 647--654, 2014.Google ScholarDigital Library
- C.W Pui, G Chen, Y Ma, EFY Young, and B Yu. Clock-aware ultrascale FPGA placement with machine learning routability prediction. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pages 929--936, 2017.Google ScholarCross Ref
- Wuxi Li, Shounak Dhar, and David Z. Pan. UTPlaceF: A routability-driven FPGA placer with physical and congestion aware packing. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2016.Google ScholarDigital Library
- Wuxi Li, Meng Li, Jiajun Wang, and David Z. Pan. UTPlaceF 3.0: A parallelization framework for modern FPGA global placement. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD) .Google Scholar
- Chang Xu, Gai Liu, Ritchie Zhao, Stephen Yang, Guojie Luo, and Zhiru Zhang. A parallel bandit-based approach for autotuning FPGA compilation. In International Symposium on FPGA, pages 157--166, 2017.Google ScholarDigital Library
- Chak-Wa Pui, Gengjie Chen, Yuzhe Ma, Evangeline F. Y. Young, and Bei Yu. Clock-aware ultrascale FPGA placement with machine learning routability prediction. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2017.Google ScholarCross Ref
- G.-J. Nam. ISPD 2006 Placement Contest: Benchmark suite and results. In Proc. Int. Symp. on Physical Design, pages 167--167, April 2006.Google ScholarDigital Library
- S. Dolgov, A. Volkov, L. Wang, and B. Xu. 2019 CAD Contest: LEF/DEF based global routing. In Proc. Int. Conf. on Computer Aided Design, pages 1--4, November 2019.Google ScholarCross Ref
- IEEE CEDA Design Automation Technical Committee. https://ieee-ceda.org/node/2591.Google Scholar
- J. Jung, I. H.-R. Jiang, G.-J. Nam, V. N. Kravets, L. Behjat, and Y.-L. Li. OpenDesign Flow Database: The infrastructure for VLSI design and design automation research. In Proc. Int. Conf. on Computer Aided Design, pages 42:1--42:6, November 2016.Google ScholarDigital Library
- J. Jung, P.-Y. Lee, Y. Wu, N. K. Darav, I. H. Jiang, V. N. Kravets, L. Behjat, Y. Li, and G.-J. Nam. DATC RDF: Robust design flow database. In Proc. Int. Conf. on Computer Aided Design, pages 872--873, November 2017.Google Scholar
- J. Jung, I. H.-R. Jiang, J. Chen, S.-T. Lin, Y.-L. Li, V. N. Kravets, and G.-J. Nam. DATC RDF: An academic flow from logic synthesis to detailed routing. In Proc. Int. Conf. on Computer Aided Design, pages 37:1--37:4, November 2018.Google ScholarDigital Library
- J. Jung, I. H.-R. Jiang, J. Chen, S.-T. Lin, Y.-L. Li, V. N. Kravets, and G.-J. Nam. DATC RDF: An open design flow from logic synthesis to detailed routing. In Proc. Workshop Open-Source EDA Tech. (WOSET), pages 6:1--6:4, November 2018.Google Scholar
- J. Chen, I. H.-R. Jiang, J. Jung, A. B. Kahng, V. N. Kravets, Y.-L. Li, S.-T. Lin, and M. Woo. DATC RDF-2019: Towards a complete academic reference design flow. In Proc. Int. Conf. on Computer Aided Design, pages 1--6, November 2019.Google ScholarCross Ref
- J. Chen, I. H.-R. Jiang, J. Jung, A. B. Kahng, V. N. Kravets, Y.-L. Li, S.-T. Lin, and M. Woo. DATC RDF-2020: Strengthening the foundation for academicresearch in ic physical design. In Proc. Int. Conf. on Computer Aided Design, pages 1--6, November 2020.Google ScholarDigital Library
- DATC RDF Calibrations. https://github.com/ieee-ceda-datc/datc-rdf-calibrations.Google Scholar
- T. Ajayi, V. A. Chhabria, M. Fogaca, S. Hashemi, A. Hosny, A. B. Khang, M. Kim, J. Lee, U. Mallappa, M. Neseem, G. Pradipta, S. Reda, M. Saligane, S. S Sapatnekar, C. Sechen, M. Shalan, W. Swartz, L. Wang, Z. Wang, M. Woo, and B. Xu. Toward an open-source digital flow: First learnings from the OpenROAD project. In Proc. Design Automation Conf, pages 76:1 -- 76:4, 2019.Google ScholarDigital Library
Index Terms
- Still Benchmarking After All These Years
Recommendations
Benchmarking in digital circuit design automation
This paper focuses on benchmarking, which is the main experimental approach to the design method and EDA-tool analysis, characterization and evaluation. We discuss the importance and difficulties of benchmarking, as well as the recent research effort ...
From Performance to Dependability Benchmarking: A Mandatory Path
Performance Evaluation and BenchmarkingThe work on performance benchmarking has started long ago. Ranging from simple benchmarks that target a very specific system or component to very complex benchmarks for complex infrastructures, performance benchmarks have contributed to improve ...
Benchmarking in digital circuit design
MINO'08: Proceedings of the 7th WSEAS International Conference on Microelectronics, Nanoelectronics, OptoelectronicsThis paper focuses on benchmarking, which is the main experimental approach to the design method and EDA-tool analysis, characterization and evaluation. We discuss the importance and difficulties of benchmarking, as well as the recent research effort ...
Comments