skip to main content
10.1145/2188286.2188309acmconferencesArticle/Chapter ViewAbstractPublication PagesicpeConference Proceedingsconference-collections
research-article

Towards efficient supercomputing: searching for the right efficiency metric

Published: 22 April 2012 Publication History

Abstract

Efficiency in supercomputing has traditionally focused on execution time. In early 2000's, the concept of total cost of ownership was re-introduced, with the introduction of efficiency measure to include aspects such as energy and space. Yet the supercomputing community has never agreed upon a metric that can cover these aspects completely and also provide a fair basis for comparison. This paper examines the metrics that have been proposed in the past decade, and proposes a vector-valued metric for efficient supercomputing. Using this metric, the paper presents a study of where the supercomputing industry has been and where it stands today with respect to efficient supercomputing.

References

[1]
L. Barroso and U. Hölzle. The case for energy-proportional computing. IEEE Computer, 40(12):33--37, Dec. 2007.
[2]
C. Bekas and A. Curioni. A new energy aware performance metric. In International Conference on Energy-Aware High Performance Computing, Sept. 2010.
[3]
K. D. Bois, T. Schaeps, S. Polfliet, F. Ryckbosch, and L. Eeckhout. SWEEP: Evaluating computer system energy efficiency using synthetic workloads. In International Conference on High Performance Embedded Architectures and Compilers, Jan. 2011.
[4]
D. Brooks, P. Bose, S. Schuster, H. Jacobson, P. Kudva, A. Buyuktosunoglu, J.-D. Wellman, V. Zyuban, M. Gupta, and P. Cook. Power-aware microarchitecture: Design and modeling challenges for the next generation microprocessors. IEEE Micro, 20(6):26--44, November/December 2000.
[5]
T. Conte and W. Hwu. Advances in benchmarking techniques: New standards and quantitative metrics. Advances in Computers, 41:231--253, 1995.
[6]
J. Dongarra, P. Luszczek, and A. Petitet. The LINPACK benchmark: Past, present, and future. Concurrency and Computation: Practice and Experience, 15(9):803--820, Aug. 2003.
[7]
Energy Efficient High Performance Computing Working Group. http://eehpcwg.lbl.gov.
[8]
A. Fanara, E. Haines, and A. Howard. The state of energy and performance benchmarking for enterprise servers. In TPC Technology Conference on Performance Evaluation and Benchmarking, Aug. 2009.
[9]
R. Ge, X. Feng, and K. Cameron. Improvement of power-performance efficiency for high-end computing. In Workshop on High-Performance, Power-Aware Computing, Apr. 2005.
[10]
R. Gonzalez and M. Horowitz. Energy dissipation in general purpose microprocessors. IEEE Journal of Solid-State Circuits, 31(9):1277--1284, Sept. 1996.
[11]
The Graph 500 list. http://www.graph500.org.
[12]
L. Gray, A. Kumar, and H. Li. Workload characterization of the SPECpower_ssj2008 benchmark. In SPEC International Performance Evaluation Workshop, June 2008.
[13]
The Green500 list: Environmentally responsible supercomputing. http://www.green500.org.
[14]
D. Hackenberg, R. Schöne, D. Molka, M. Müller, and A. Knüpfer. Quantifying power consumption variations of HPC systems using SPEC MPI benchmarks. In International Conference on Energy-Aware High Performance Computing, Sept. 2010.
[15]
HPC Challenge benchmark. http://icl.cs.utk.edu/hpcc/.
[16]
C. Hsu, W. Feng, and J. Archuleta. Towards efficient supercomputing: A quest for the right metric. In Workshop on High-Performance, Power-Aware Computing, Apr. 2005.
[17]
J. Koomey, S. Berard, M. Sanchez, and H. Wong. Implications of historical trends in the electrical efficiency of computing. IEEE Annals of the History of Computing, 33(3):46--54, July-September 2011.
[18]
K.-D. Lange and M. Tricker. The design and development of the server efficiency rating tool (SERT). In International Conference on Performance Engineering, Mar. 2011.
[19]
T. Makimoto, K. Eguchi, and M. Yoneyama. The cooler the better: New directions in the nomadic age. IEEE Computer, 34(4):38--42, Apr. 2001.
[20]
A. Martin. Towards an energy complexity of computation. Information Processing Letters, 77:181--187, Feb. 2001.
[21]
D. Meisner and T. Wenisch. Does low-power design imply energy efficiency for data centers. In International Symposium on Low Power Electronics and Design, Aug. 2011.
[22]
S. Microsystems. The SWaP (Space, Watts and Performance) metric, 2007.
[23]
D. Molka, D. Hackenberg, R. Schöne, T. Minartz, and W. Nagel. Flexible workload generation for HPC cluster efficiency benchmarking. In International Conference on Energy-Aware High Performance Computing, Sept. 2011.
[24]
T. Mudge and U. Hölzle. Challenges and opportunities for extremely energy-efficient processors. IEEE Micro, 30(4):20--24, July/August 2010.
[25]
M. Müller, M. van Waveren, R. Lieberman, B. Whitney, H. Saito, K. Kumaran, J. Baron, W. Brantley, C. Parrott, T. Elken, H. Feng, and C. Ponder. SPEC MPI2007:an application benchmark suite for parallel systems using MPI. Concurrency and Computation: Practice and Experience, 22(2):191--205, Feb. 2010.
[26]
V. Pel. Energy efficiency aspects in Cray supercomputers. In International Conference on Energy-Aware High Performance Computing, Sept. 2010.
[27]
P. Pénzes and A. Martin. Energy-delay efficiency of VLSI computations. In ACM Great Lakes Symposium on VLSI, Apr. 2002.
[28]
M. Poess, R. Nambiar, and K. Vaid. Optimizing benchmark configurations for energy efficiency. In International Conference on Performance Engineering, Mar. 2011.
[29]
M. Poess, R. Nambiar, K. Vaid, J. J. M. Stephens, K. Huppler, and E. Haines. Energy benchmarks: A detailed analysis. In International Conference on Energy-Efficient Computing and Networking, Apr. 2010.
[30]
S. Rivoire, M. Shah, P. Ranganathan, and C. Kozyrakis. JouleSort: A balanced energy-efficiency benchmark. In International Conference on Management of Data, June 2007.
[31]
F. Ryckbosch, S. Polfliet, and L. Eeckhout. Trend in server energy-proportionality. IEEE Computer, 44(9):69--72, Sept. 2011.
[32]
V. Sarkar. ExaScale software study: Software challenges in extreme scale systems. DARPA IPTO Report, Sept. 2009.
[33]
D. Schall, V. Hoefner, and M. Kern. Towards an enhanced benchmark advocating energy-efficient systems. In TPC Technology Conference on Performance Evaluation & Benchmarking, Aug. 2011.
[34]
T. Scogland, B. Subramaniam, and W. Feng. Emerging trends on the evolving Green500: Year three. In Workshop on High-Performance, Power-Aware Computing, May 2011.
[35]
SPECpower_ssj2008 Results. http://www.spec.org/power_ssj2008/results/.
[36]
E. Strohmaier. Generalized utility metrics for supercomputers. June 2009.
[37]
The Green Grid. The Green Grid power efficiency metrics: PUE and DCiE, 2007.
[38]
The Green Grid. Recommendations for measuring and reporting overall data center efficiency (version 2), May 2011.
[39]
TOP500 supercomputing sites. http://www.top500.org.
[40]
G. Varsamopoulos and S. Gupta. Energy proportionality and the future: Metrics and directions. In International Workshop on Green Computing, Sept. 2010.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICPE '12: Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering
April 2012
362 pages
ISBN:9781450312028
DOI:10.1145/2188286
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 April 2012

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. energy efficiency
  2. green500
  3. specpower
  4. top500

Qualifiers

  • Research-article

Conference

ICPE'12
Sponsor:

Acceptance Rates

Overall Acceptance Rate 252 of 851 submissions, 30%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)1
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2020)Finally, how many efficiencies the supercomputers have?The Journal of Supercomputing10.1007/s11227-020-03210-4Online publication date: 27-Feb-2020
  • (2019)Energy-Aware High-Performance ComputingScientific Programming10.1155/2019/83487912019Online publication date: 1-Jan-2019
  • (2016)The Right Metric for Efficient Supercomputing: A Ten-Year Retrospective2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)10.1109/IPDPSW.2016.82(1090-1093)Online publication date: May-2016
  • (2016)Chip-level and multi-node analysis of energy-optimized lattice Boltzmann CFD simulationsConcurrency and Computation: Practice & Experience10.1002/cpe.348928:7(2295-2315)Online publication date: 1-May-2016
  • (2015)Measuring Server Energy ProportionalityProceedings of the 6th ACM/SPEC International Conference on Performance Engineering10.1145/2668930.2688049(235-240)Online publication date: 28-Jan-2015
  • (2015)ALEAProceedings of the 2015 International Conference on Parallel Architecture and Compilation (PACT)10.1109/PACT.2015.16(87-98)Online publication date: 18-Oct-2015
  • (2015)Technological forecasting of supercomputer development: The March to Exascale computingOmega10.1016/j.omega.2014.09.00951(128-135)Online publication date: Mar-2015
  • (2013)Campus HPC Network Design and Monitoring2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing10.1109/HPCC.and.EUC.2013.212(1504-1511)Online publication date: Nov-2013

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media