Skip to main content
SpringerLink
Log in

DynBench: A dynamic benchmark suite for distributed real-time systems

  • Conference paper
  • First Online:
Parallel and Distributed Processing (IPPS 1999)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1586))

Included in the following conference series:

Abstract

In this paper we present the architecture and framework for a benchmark suite that has been developed as part of the DeSiDeRaTa project. The proposed benchmark suite is representative of the emerging generation of distributed, mission-critical, real-time control systems that operate in dynamic environments. Systems that operate in such environments may have unknown worst-case scenarios, may have large variances in the sizes of the data and event sets that they process (and thus, have large variances in execution latencies and resource requirements), and may be very difficult to characterize statically, even by time-invariant statistical distributions. The proposed benchmark suite (called DynBench) is useful for evaluation of the Quality of Service (QoS) management and/or Resource Management (RM) services in distributed real-time systems. As such, DynBench includes a set of performance metrics for the evaluation of the QoS and RM technologies in dynamic distributed real-time systems. The paper demonstrates the successful application of DynBench in evaluation of the DeSiDeRaTa QoS management middle-ware.

Sponsored in part by DARPA/NCCOSC contract N66001-97-C-8250, and by NSWC/NCEE contracts NCEE/A303/41E-96 and NCEE/A303/50A-98.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • [CS+97] S. Chatterjee, J. Sydir, B. Sabata, and T. Lawrence, “Modeling Applications for Adaptive QoS-based Resource Management,” 2nd IEEE High-Assurance System Engineering Workshop (HASE97), Bethesda, Maryland, August 1997.

    Google Scholar 

  • [Cur76] H.J. Curnow and B.A. Wichmann, “A synthetic benchmark,” Computer Journal 19(1), January 1976, 43–49.

    Article  Google Scholar 

  • [Dav80] C.G. Davis, “Ballistic missile defense: a supercomputer challenge,” IEEE Computer, 30, 1980, 37–46.

    Google Scholar 

  • [DL97] Z. Deng and J. W.-S. Liu, “Scheduling real-time applications in an open environment,” Real-Time Systems Symposium, San Francisco, California, December 1997, pp. 308–319.

    Google Scholar 

  • [Har94] Robert D. Harrison Jr., “Combat system prerequisites on supercomputer performance analysis,” in Proceedings of the NATO Advanced Study Institute on Real Time Computing, NATO ASI Series F(127), 512–513, Springer-Verlag 1994.

    Google Scholar 

  • [Hon97] Honeywell Technology, “Adaptive Computing systems Benchmark,” http://www.htc.honeywell.com/projects/acsbench/.

    Google Scholar 

  • [H+97] J. Huang, etal., “RT-ARM: A real-time adaptive resource management system for distributed mission-critical applications,” Workshop on Middleware for Distributed Real-Time Systems, RTSS-97.

    Google Scholar 

  • [Kam91] N.I. Kamenoff and N.H. Weiderman, “Hartstone distributed benchmark: requirements and definitions,” in Proceedings of the 12 th IEEE Real-Time Systems Symposium, 1991, 199–208.

    Google Scholar 

  • [Kar89] Kar, R.P. and Porter, K., “Rhealstone—A Real Time Benchmarking Proposal,” Dr. Dobbs Journal 14(2):4–24, February, 1989

    Google Scholar 

  • [Kav92] K.M. Kavi and S.M. Yang, “Real-time systems design methodologies: an introduction and a survey,” The Journal of Systems and Software, April 1992, 85–99, Elsevier Science Publishers.

    Google Scholar 

  • [Kis94] D.L. Kiskis and K.G. Shin, “SWSL: A synthetic workload specification language for real-time systems,” IEEE Transactions on Software Engineering, 20(10), Oct. 1994, 798–811.

    Article  Google Scholar 

  • [Kis96] D.L. Kiskis and K.G. Shin, “A synthetic workload for a distributed real-time system,” Journal of Real-Time Systems, 11(1), July 1996, 5–18.

    Article  Google Scholar 

  • [Koob98] G. Koob, “DARPA, Quorum mission statement,” http://www.darpa.mil/ito/research/quorum/index.html.

    Google Scholar 

  • [Mast95] Michael W. Masters, “Real-time Computing Cornerstones: A System Engineer’s View,” Proceedings of the 3rd Workshop on Parallel and Distributed Real-time Systems, 1995

    Google Scholar 

  • [Met96] R.C. Metzger, B. VanVoorst, L.S. Pires, R. Jha, W. Au, M. Amin,D.A. Castanon and V. Kumar, “C3I parallel benchmark suite: introduction and preliminary results,” in Supercomputing’96, 1996.

    Google Scholar 

  • [Mil95] D.L. Mills, “Improved Algorithms for Synchronizing Computer Network Clocks,” IEEE Trans. on Networks, 3, June 1995.

    Google Scholar 

  • [Mol90] J.J. Molini, S.K. Maimon and P.H. Watson, “Real-time system scenarios,” in Proceedings of The 11 th IEEE Real-Time Systems Symposium, 214–225, IEEE Computer Society Press, 1990.

    Google Scholar 

  • [Pri89] W.J. Price, “A benchmark tutorial,” IEEE Micro, October 1989, 28–43.

    Google Scholar 

  • [RLLS97] R. Rajkumar, C. Lee, J. Lehoczky, and D. Siewiorek, “A resource allocation model for qos management,” 18 th IEEE Real-Time Systems Symposium, pages 298–307, 1997.

    Google Scholar 

  • [Ravin98a] B. Ravindran, L.R. Welch, and B. Shirazi, “A Resource Management Middleware for Dynamic, Dependable Real-Time Systems,” Tech Report, CSE Dept, UTA, Aug. 1998.

    Google Scholar 

  • [Ravin98b] B. Ravindran, L.R. Welch, C. Bruggeman, B. Shirazi, C. Cavanaugh, “A Resource Management Model for Dynamic, Scalable, Dependable, Real-Time Systems,” Lecture Notes in Computer Science, #1388, Springer Verlag, March 1998, pp. 931–936.

    Google Scholar 

  • [RSYJ97] D. Rosu, K. Schwan, S. Yalamanchili, and R. Jha, “On adaptive resource allocation for complex real-time applications,” In Proceedings of the 18 th IEEE Real-Time Systems Symposium, 1997, 320–329.

    Google Scholar 

  • [Ujv97] B.G. Ujvary and N.I. Kamenoff, “Implementation of the Hartstone distributed benchmark for real-time distributed systems: results and conclusions,” in Proceedings of the 5 th International Workshop on Parallel and Distributed Real-Time Systems, 98–103, IEEE Computer Society Press, 1997.

    Google Scholar 

  • [Wei84] R.P. Weicker, “Dhrystone: a synthetic systems programming benchmark,” Communications of the ACM, 27(10), 1984, 1013–1030.

    Article  Google Scholar 

  • [Wei89] N.H. Weiderman, “Hartstone: synthetic benchmark requirements for hard real-time applications,” CMU/SEI-89-TR-23, June 89.

    Google Scholar 

  • [Wei90] R.P. Weicker, “An overview of common benchmarks,” IEEE Computer, 23(12), Dec. 1990, 65–75.

    Google Scholar 

  • [Wei92] N.H. Weiderman and N.I. Kamenoff, “Hartstone uniprocessor benchmark: definitions and experiments for real-time systems,” Journal of Real-Time Systems, 4(4), Dec. 1992, 353–382.

    Article  Google Scholar 

  • [Wel96] L. R. Welch, Binoy Ravindran, Robert D. Harrison, Leslie Madden, Michael W. Masters and Wayne Mills, “Challenges in Engineering Distributed Shipboard Control Systems,” in Proceedings of Work-in-Progress Session of The IEEE Real-Time Systems Symposium, December 1996, 19–22.

    Google Scholar 

  • [Welch97a] L.R. Welch and B. Shirazi, “DeSiDeRaTa: QoS Management Tools for Dynamic, Scalable, Dependable, Real-Time Systems,” IEEE Workshop on Middleware for Distributed Real-Time Systems and Services, Dec. 1997, pp. 164–178.

    Google Scholar 

  • [Welch98a] L.R. Welch, B. Ravindran, B. Shirazi, C. Bruggeman, “Specification and Modeling of Dynamic, Distributed Real-time Systems,” Proceedings of the 19 th IEEE Real-Time Systems Symposium (RTSS’98), Dec. 1998, pp. 72–81.

    Google Scholar 

  • [Welch98b] L.R. Welch, B. Shirazi, B. Ravindran, C. Bruggeman, “DeSiDeRaTa: QoS Management Technology for Dynamic, Scalable, Dependable, Real-time Systems,” 15th IFAC Workshop—Distributed Computer Control Systems (DCCS’98), Sept. 1998.

    Google Scholar 

  • [Welch98c] L.R. Welch, et al., “Adaptive Resource Management for Scalable, Dependable Real-time Systems,” in the proceedings of the Work-In-Progress Session of the Fourth IEEE Real-Time Technology and Applications Symposium (RTAS’98), June 1998, pp. 3–6.

    Google Scholar 

  • [Welch98d] L.R. Welch and B. Shirazi, “A Distributed Architecture for QoS Management of Dynamic, Scalable, Dependable, Real-Time Systems,” Workshop on Parallel and Distributed Real-Time Systems, April 1998, available on 1998 IPPS/SPDP CD-ROM.

    Google Scholar 

  • [Welch98e] L.R. Welch, B. Shirazi, B. Ravindran, and F. Kamangar, “Instrumentation, Modeling, and Analysis of Dynamic, Distributed Real-time Systems,” Tech Rep, CSE Dept, UTA, 1998.

    Google Scholar 

  • [ZBS97] John A. Zinky, David E. Bakken, and Richard Schantz, “Architectural Support for Quality of Service for CORBA Objects,” Theory and Practice of Object Systems, 1997.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Texas at Arlington, Box 19015, 76019-0015, Arlington, TX

    Behrooz Shirazi, Lonnie Welch, Charles Cavanaugh, Bharath Yanamula, Russ Brucks & Eui-nam Huh

  2. The Bradley Dept. of Electrical and Computer Engineering, Virginia Tech, 340 Whittemore Hall, Mail Code 0111, 24061, Blacksburg, VA

    Binoy Ravindran

Authors
  1. Behrooz Shirazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lonnie Welch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Binoy Ravindran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Charles Cavanaugh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bharath Yanamula
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Russ Brucks
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Eui-nam Huh
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

José Rolim Frank Mueller Albert Y. Zomaya Fikret Ercal Stephan Olariu Binoy Ravindran Jan Gustafsson Hiroaki Takada Ron Olsson Laxmikant V. Kale Pete Beckman Matthew Haines Hossam ElGindy Denis Caromel Serge Chaumette Geoffrey Fox Yi Pan Keqin Li Tao Yang G. Chiola G. Conte L. V. Mancini Domenique Méry Beverly Sanders Devesh Bhatt Viktor Prasanna

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer-Verlag

About this paper

Cite this paper

Shirazi, B. et al. (1999). DynBench: A dynamic benchmark suite for distributed real-time systems. In: Rolim, J., et al. Parallel and Distributed Processing. IPPS 1999. Lecture Notes in Computer Science, vol 1586. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0098013

Download citation

  • DOI: https://doi.org/10.1007/BFb0098013

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-65831-3

  • Online ISBN: 978-3-540-48932-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation