Skip to main content
SpringerLink
Log in

The Design, Performance, and Use of DiPerF: An automated DIstributed PERformance evaluation Framework

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

We present DiPerF, a DIstributed PERformance evaluation Framework, aimed at simplifying and automating performance evaluation of networked services. DiPerF coordinates a pool of machines that access a target service and collect performance measurements, aggregates these measurements, and generates performance statistics. The aggregate data collected provide information on service throughput, service response time, service ‘fairness’ when serving multiple clients concurrently, and on the impact of network connectivity on service performance. We have used DiPerF in various environments (PlanetLab, Grid3, TeraGrid, and a cluster) and with a large number of services. This paper provides data that demonstrates that DiPerF is accurate: The aggregate client view matches the tested service view within a few percents, and scalable: DiPerF can handle more than 10,000 clients and 100,000 transactions per second. Moreover, rapid adoption and extensive use demonstrate that the ability to automate performance characteristics extraction makes DiPerF a valuable tool.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Peterson, L., Anderson, T., Culler, D., Roscoe, T.: A blueprint for introducing disruptive technology into the internet. In: Proceedings of the First ACM Workshop on Hot Topics in Networking (HotNets), October 2002

  2. Foster, I., et al.: The Grid2003 production grid: Principles and practice. In: 13th IEEE Intl. Symposium on High Performance Distributed Computing, 2004

  3. Foster, I., Kesselman, C.: The Grid 2: Blueprint for a New Computing Infrastructure, Chapter 1: Perspectives. Elsevier (2003)

  4. Dumitrescu, C., Foster, I.: GRUBER: A Grid Resource SLA-based Broker. EuroPar (2005)

  5. Dumitrescu, C., Raicu, I., Foster, I.: DI-GRUBER: A Distributed Approach for Grid Resource Brokering. SC (2005)

  6. Raicu, I.: A performance study of the Globus Toolkit® and Grid Services via DiPerF, an automated DIstributed PERformance testing framework. University of Chicago, Computer Science Department, MS thesis, May 2005

  7. Raicu, I.: Decreasing end-to-end job execution times by increasing resource utilization using predictive scheduling in the Grid. Technical report, Grid Computing Seminar, Department of Computer Science, University of Chicago, March 2005

  8. Dumitrescu, C., Raicu, I., Foster, I.: Experiences in running workloads over Grid3”. The 4th International Conference on Grid and Cooperative Computing (GCC’05), Beijing China, December, 2005

  9. Allcock, B., Bresnahan, J., Kettimuthu, R., Link, M., Dumitrescu, C., Raicu, I., Foster, I.: Zebra: The Globus Striped GridFTP Framework and Server. sc, p. 54, ACM/IEEE SC 2005 Conference (SC’05), 2005

  10. Dumitrescu, C., Raicu, I., Ripeanu, M., Foster, I.: DiPerF: An automated DIstributed PERformance testing Framework. 5th International IEEE/ACM Workshop in Grid Computing, Pittsburg, PA, 2004

  11. Internet Protocol Performance Metrics. http://www.advanced.org/ippm/index.html, 1998

  12. Paxson, V., Almes, G., Mahdavi, J., Mathis, M.: Framework for IP performance metrics. IETF RFC 2330, 1998

  13. The Surveyor Project. http://www.advanced.org/csgippm/, 1998

  14. Keynote Systems Inc. http://www.keynote.com, 1998

  15. Steigner, Ch., Wilke, J.: Isolating performance bottlenecks in network applications. In: Proceedings of the International IPSI-2003 Conference, Sveti Stefan, Montenegro, October 4–11, 2003

  16. Lowekamp, B.B., Miller, N., Karrer, R., Gross, T., Steenkiste, P.: Design, implementation, and evaluation of the Remos Network Monitoring System. Journal of Grid Computing 1(1), 75–93 (2003)

    Article  Google Scholar 

  17. Barford, P., Crovella, M.E.: Measuring Web performance in the wide area. Performance Evaluation Review, Special Issue on Network Traffic Measurement and Workload Characterization, August 1999

  18. Banga, G., Druschel, P.: Measuring the capacity of a Web server under realistic loads. World Wide Web Journal (Special Issue on World Wide Web Characterization and Performance Evaluation), 1999

  19. Gunter, D., Tierney, B., Tull, C.E., Virmani, V.: On-Demand Grid Application Tuning and Debugging with the NetLogger Activation Service, 4th International Workshop on Grid Computing, Grid2003, Phoenix, Arizona, November 17th, 2003

  20. Tsouloupas, G., Dikaiakos, M.: GridBench: A tool for benchmarking grids. 4th International Workshop on Grid Computing, Grid2003, Phoenix, Arizona, November 17th, 2003

  21. The Globus Alliance. Globus Toolkit 3.0 Test Results Page. http://www-unix.globus.org/ogsa/tests/gt3_tests_result.html

  22. The Globus Alliance: Overview and Status of Current GT Performance Studies. http://www-unix.globus.org/toolkit/docs/development/3.9.5/perf_overview.html

  23. Lee, C., Wolski, R., Foster, I., Kesselman, C., Stepanek, J.: A Network Performance Tool for Grid Environments, SC ’99

  24. Wolski, R., Spring, N., Hayes, J.: The network weather service: A distributed resource performance forecasting service for metacomputing. Future Generation Computing Systems, 1999

  25. select(2) – Linux man page. http://www.die.net/doc/linux/man/man2/select.2.html

  26. Chandra, A., Mosberger, D.: Scalability of Linux event-dispatch mechanisms. In: Proceedings of the USENIX Annual Technical Conference (USENIX 2001), Boston, Massachusetts, June 2001

  27. Minar, N.: A survey of the NTP protocol. MIT Media Lab, 1999, http://xenia.media.mit.edu/~nelson/research/ntp-survey99

  28. Williams, T., Kelley, C.: gnuplot, an interactive plotting program. http://www.gnuplot.info/docs/gnuplot.pdf

  29. Massie, M.L., Chun, B.N., Culler, D.E.: The ganglia distributed monitoring system: Design, implementation, and experience. Parallel Comput. 30(7), (July 2004)

  30. Grid3. http://www.ivdgl.org/grid3/

  31. Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., Bowman, M.: PlanetLab: An overlay testbed for broad-coverage services. ACM Computer Communications Review, vol. 33, no. 3, July 2003

  32. Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid information services for distributed resource sharing. In: Proceedings of the Tenth IEEE International Symposium on High-Performance Distributed Computing (HPDC-10). IEEE (August 2001)

  33. TeraGrid, http://www.teragrid.org/

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, The University of Chicago, Chicago, IL, USA

    Ioan Raicu & Ian Foster

  2. Electrical Engineering, Mathematical and Computer Science Department, Delft University of Technology, Delft, The Netherlands

    Catalin Dumitrescu

  3. Electrical and Computer Engineering Department, The University of British Columbia, Victoria, BC, USA

    Matei Ripeanu

  4. Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, USA

    Ian Foster

Authors
  1. Ioan Raicu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Catalin Dumitrescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matei Ripeanu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ian Foster
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ioan Raicu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Raicu, I., Dumitrescu, C., Ripeanu, M. et al. The Design, Performance, and Use of DiPerF: An automated DIstributed PERformance evaluation Framework. J Grid Computing 4, 287–309 (2006). https://doi.org/10.1007/s10723-006-9037-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-006-9037-5

Key words

Search

Navigation