Abstract
Microprocessor design teams use a combination of simulation-based and formal verification techniques to validate the pre-silicon models prior to “tape-out” and chip fabrication. Pseudo-random test case generation to “cover” the architectural space is still relied upon as the principal means to identify design bugs. However, such methods are limited to functional bugs only. Detection and diagnosis of timing (performance) bugs at the architectural level is largely an expert job. Architects guide the performance team to run manually generated test cases to validate the design from a performance viewpoint. In this paper, we will review some of the new approaches being tried out to automate the generation of performance test cases. We will show how this can be done within the basic framework of current functional validation and testing of pre-silicon processor models. Three categories of “reference” specifications are used in determining the defect-free pipeline timing behavior associated with generated test cases: (a) axiomatic specifications of intrinsic machine latencies and bandwidths; (b) proven analytical models for simple basic block and loop test cases; and, (c) a stable reference behavioral/functional (pre-RTL) model of the processor under development. We report experimental results obtained in performance validation studies applied to real PowerPC™ processor development projects.
Similar content being viewed by others
References
A. Kuehlmann, A. Srinivasan, and D.P. LaPotin, “Verity-A Formal Verification Program for Custom CMOS Circuits,” IBM Journ. Res & Develop., Vol. 39, No. 1/2, pp. 149–165, Jan./March 1995.
I. Beer, S. Ben-David, C. Eisner, D. Geist, L. Gluhovosky, T. Heyman, A. Landver, P. Pannah, Y. Rodeh, G. Ronin, and Y. Wolfsthal, “Rulebase: Model Checking in IBM,” Computer-Aided Verification (CAV '97). Lecture Notes in Computer Science, Vol. 1254, 1997, pp. 480–483, Springer Verlag.
A. Aharon, A. Bar-David, B. Dorfman, E. Gofman, M. Leibowitz, and V. Schwatzburd, “Verification of the IBM RISC System/6000 by a Dynamic-Biased Pseudo-Random Test Program Generator,” IBM Systems Journal, Vol. 30, No. 4, April 1991.
A. Chandra, V. Iyengar, D. Jameson, R. Jawalekar, I. Nair, B. Rosen, M. Mullen, J. Yoon, R. Armoni, D. Geist, and Y. Wolfsthal, “AVPGEN-A Test Generator for Architecture Verification,” IEEE Trans. on VLSI Systems, Vol. 3, No. 2, pp. 188–200, June 1995.
M. Kantrowitz and L.M. Noack, “Functional Verification of a Multiple-Issue, Pipelined, Superscalar Alpha Processor-The Alpha 21164 CPU Chip,” Proc. Digital Tech. Journal, 1995, Vol. 7, No. 1, pp. 136–144.
A. Aharon, D. Goodman, M. Levinger, Y. Lichtenstein, Y. Malka, C. Metzger, M. Molcho, and G. Shurek, “Test Program Generation for Functional Verification of PowerPC Processors in IBM,” Proc. 32nd. ACM/IEEE Design Automation Conference (DAC), 1995, pp. 279–285.
X. Shen and Arvind, “Modeling and Verification of ISA Implementations,” Proc. Australasian Computer Architecture Conference, Perth, Australia, Feb. 1998. See also IEEE Micro, Vol. 19, No. 3, May/June 1999, pp. 36–46.
P. Bose, “Architectural Timing Verification and Test for Super Scalar Processors,” Proc. of 24th Fault-Tolerant Computing Symposium, FTCS-24, June 1994, pp. 256–265.
P. Bose and S. Surya, “Architectural Timing Verification and Test for Super Scalar Processors,” IBM Journ. Res. & Develop., Vol. 39, Nos. 1/2, pp. 113–129, Jan./March 1995.
P. Bose and T.M. Conte, “Performance Analysis and its Impact on Design,” IEEE Computer, Vol. 31, No. 5, pp. 41–49, May 1998.
M. Reilly and J. Edmondson, “Performance Simulation of an Alpha Microprocessor,” IEEE Computer, Vol. 31, No. 5, pp. 50–58, May 1998.
A. Poursepanj, “The PowerPC Performance Modeling Methodology,” Comm. of the ACM, Vol. 37, No. 6, pp. 47–55, June 1994.
B. Black and J.P. Shen, “Calibration of Microprocessor Performance Models,” IEEE Computer, Vol. 31, No. 5, pp. 59–75, May 1998.
C. May, E. Silha, R. Simpson, and H. Warren (Eds.), The PowerPC TM Architecture: A Specification for a New Family of RISC Processors, 2nd edition, Morgan Kaufmann Publishers, Inc., May 1994.
D. Levitan, T. Thomas, and P. Tu, “The PowerPC 620 MicroprocessorTM: A High Performance Superscalar RISC Microprocessor,” Proc. IEEE Spring COMPCON, 1995.
T.A. Diep, C. Nelson, and J.P. Shen, “Performance Evaluation of the PowerPC 620 Microarchitecture,” Proc. 22nd Int'l. Symp. on Computer Architecture (ISCA), June 1995.
P. Song, “IBM's POWER3 to Replace P2SC,” Microprocessor Report, August 26, 1997; see also, article by BYTE senior editor T.R. Halfhill, “IBM's Powerhouse Chip,” in BYTE, pp. 51–52, April 1998.
W. Mangione-Smith, T.-P. Shieh, S.G. Abraham, and E.S. Davidson, “Approaching a Machine Application-Bound in Delivered Performance on Scientific Code,” Proc. IEEE, Aug. 1993, Vol. 81, pp. 1166–1178.
G.F. Grohoski, “Machine Organization of the IBM RISC System/6000 Processor,” IBM J. Res. & Develop., Vol. 36, No. 1, pp. 37–58, Jan. 1990.
P. Bose, “Performance Test Case Generation for Microprocessors,” Proc. 16th IEEE VLSI Test Symposium, Monterey, CA, April 1998, pp. 54–59.
P. Bose, S. Kim, F.P. O'Connell, and W.A. Ciarfella, “Bounds Modelling and Compiler Optimizations for Super Scalar Performance Tuning,” Journal of Systems Architecture, Vol. 45, pp. 1111–1137, 1999.
N. Utamaphethai, R.D. Blanton, and J.P. Shen, “Superscalar Processor Validation at the Microarchitecture Level,” Digest of Papers of the International High Level Design Validation and Test Workshop (HLDVT), pp. 202–209, Nov. 1997.
P. Bose and J.A. Abraham, “Fault Modeling for Performance Test Case Generation,” Proc. 2nd. Int'l. Workshop on Microprocesssor Test and Verification, (held in conjunction with ITC'99), Atlantic City, Oct. 1999.
J.H. Moreno, M. Moudgill, J.-D. Wellman, P. Bose, and L. Trevillyan, “Trace-Driven Performance Exploration of a PowerPC 601 OLTP Workload on a Wide Issue Superscalar Processor,” IBM Research Report RC 20692, Yorktown Heights, NY, August 1997.
Systems Performance Evaluation Corporation (SPEC) website: http://www.specbench.org
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bose, P. Testing for Function and Performance: Towards an Integrated Processor Validation Methodology. Journal of Electronic Testing 16, 29–48 (2000). https://doi.org/10.1023/A:1008363921045
Issue Date:
DOI: https://doi.org/10.1023/A:1008363921045