Skip to main content
SpringerLink
Log in

CCMPerf: A Benchmarking Tool for CORBA Component Model Implementations

  • Published:
Real-Time Systems Aims and scope Submit manuscript

Abstract

Commercial off-the-shelf (COTS) middleware is now widely used to develop distributed real-time and embedded (DRE) systems. DRE systems are themselves increasingly combined to form “systems of systems” that have diverse quality of service (QoS) requirements. Earlier generations of COTS middleware, such as Object Request Brokers (ORBs) based on the CORBA 2.x standard, did not facilitate the separation of QoS policies from application functionality, which made it hard to configure and validate complex DRE applications. The new generation of component middleware, such as the CORBA Component Model (CCM) based on the CORBA 3.0 standard, addresses the limitations of earlier generation middleware by establishing standards for implementing, packaging, assembling, and deploying component implementations.

There has been little systematic empirical study of the performance characteristics of component middleware implementations in the context of DRE systems. This paper therefore provides four contributions to the study of CCM for DRE systems. First, we describe the challenges involved in benchmarking different CCM implementations. Second, we describe key criteria for comparing different CCM implementations using key black-box and white-box metrics. Third, we describe the design of our CCMPerf benchmarking suite to illustrate test categories that evaluate aspects of CCM implementation to determine their suitability for the DRE domain. Fourth, we use CCMPerf to benchmark CIAO implementation of CCM and analyze the results. These results show that the CIAO implementation based on the more sophisticated CORBA 3.0 standard has comparable DRE performance to that of the TAO implementation based on the earlier CORBA 2.x standard.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Corsaro, A., and Schmidt, D. C. 2002. Evaluating Real-time Java features and performance for Real-time embedded systems. In Proceedings of the 8th IEEE Real-Time Technology and Applications Symposium, IEEE, San Jose.

  • Douglas N., et al. 2004. Kansas University Real-Time (KURT) Linux, www.ittc.ukans.edu/kurt/.

  • Gamma, E., Helm, R., Johnson, R., and Vlissides, J. Design Patterns: Elements of Reusable Object-Oriented Software. MA: Addison-Wesley, Reading, 1995.

  • Gautam, T. et. al. 2003. Implementation experience with OMG’s SCIOP mapping. In Proceedings of the 5th International Symposium on Distributed Objects and Applications.

  • Gokhale, A., Schmidt, D. C., Natarajan, B., and Wang, N. 2002. Applying model-integrated computing to component middleware and enterprise applications. The Communications of the ACM Special Issue on Enterprise Components, Service and Business Rules 45(10).

  • Harrison, T. H., Levine, D. L., and Schmidt, D. C. 1997. The design and performance of a Real-time CORBA event service. In Proceedings of OOPSLA’97, Atlanta, GA: ACM, pp. 184–199.

  • Institute for Software Integrated Systems. Component Synthesis using Model Integrated Computing (CoSMIC). www.dre.vanderbilt.edu/cosmic, Vanderbilt University.

  • Krishna, A. S., Schmidt, D. C., Porter, A., Memon, A., and Sevilla-Ruiz, D. 2004a. Improving the quality of performance-intensive software via model-integrated distributed continuous quality assurance. In Proceedings of the 8th International Conference on Software Reuse, ACM/IEEE, Madrid, Spain.

  • Krishna, A. S., Yilmaz, C., Memon, A., Porter, A., Schmidt, D. C., Gokhale, A., and Natarajan, B. 2004b. Preserving distributed systems critical properties: A model-driven approach. IEEE Software Special Issue on Persistent Software Attributes.

  • L. M. A. T., Labs. 2002. ATL QoS Home Page. www.atl.external.Imco.com/projects/QoS/.

  • Memon, A., Porter, A., Yilmaz, C., Nagarajan, A., Schmidt, D. C., and Natarajan, B. 2004. Skoll: Distributed continuous quality assurance. In Proceedings of the 26th IEEE/ACM International Conference on Software Engineering. IEEE/ACM, Edinburgh, Scotland.

  • MICO. 2000. The MICO CORBA Component Project. www. fpx.de/MicoCCM/.

  • Object Management Group. 1999. Notifi Cation Sendee Specification. Object Management Group, OMG Document telecom/99-07-01 edition.

  • Object Management Group. 2001. Event Service Specification Version 1.1. OMG Document formal/01-03-01 edition.

  • Object Management Group. 2002. CORBA Components. OMG Document formal/2002-06-65 edition.

  • Object Management Group. 2003a. Data Distribution Service for Real-Time Systems Specification, 1.0 edition.

  • Object Management Group. 2003b. Deployment and Configuration Adopted Submission. OMG Document ptc/03-07-08 edition.

  • Qedo. 2002. QoS Enabled Distributed Objects. qedo.berlios.de.

  • Shevchenko, R. 2000. CORBAConf: A tool for providing autoconf support for CORBA. corbaconf.kiev.ua/.

  • Schmidt, D. C., Natarajan, B., Gokhale, A., Wang, N., and Gill, C. 2002. TAO: A pattern-oriented object request broker for distributed real-time and embedded systems. IEEE Distributed Systems Online 3(2).

  • Sharp, D. C. 1998. Reducing avionics software cost through component based product line development. In Proceedings of the 10th Annual Software Technology Conference.

  • Sharp, D. C., Pla, E., and Lueck, K. R. 2003. Evaluating real-time Java for mission-critical large-scale embedded systems. In Proceedings of the 9th IEEE Real-Time Technology and Applications Symposium, G. Bollella, (ed.), Washington D.C., pp. 30–37.

  • StarCCM. 2003. StarCCM. starccm.sourceforge.net.

  • Sztipanovits, J., and Karsai, G. 1997. Model-integrated computing. IEEE Computer 30(4): 110–112.

    Google Scholar 

  • Szyperski, C. 1998. Component Software—Beyond Object-Oriented Programming. Santa Fe, MM: Addison-Wesley.

    Google Scholar 

  • Tuma, P., and Buble, A. 2001. Open CORBA benchmarking. In International Symposium on Performance Evaluation of Computer and Telecommunication Systems.

  • Wang, N., Gill, C., Schmidt, D. C., and Subramonian, V. 2004. Confi guring Real-time aspects in component middleware. In Proceedings of the International Symposium on Distributed Objects and Applications (DOA’04), AgiaNapa, Cyprus.

  • Wang, N., Schmidt, D. C., Gokhale, A., Gill, C. D., Natarajan, B., Rodrigues, C., Loyall, J. P., and Schantz, R. E. 2003. Total quality of service provisioning in middleware and applications. The Journal of Microprocessors and Microsystems 27(2): 45–54.

    Article  CAS  Google Scholar 

  • Wang, N., Schmidt, D. C., and Vinoski, S. 1999. Collocation optimizations for CORBA. C++ Report 11(10): 47–52.

    Google Scholar 

  • White, B., and J. L. et al. 2002. An integrated experimental environment for distributed systems and networks. In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation, Boston, MA, USENIX Association, pp. 255–270.

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA

    Arvind S. Krishna, Aniruddha Gokhale & Douglas C. Schmidt

  2. Computer Science Department, Washington University, St. Louis, MO, USA

    Balachandran Natarajan & Nanbor Wang

  3. Lockheed Martin Advanced Technology Labs, Cherry Hill, NJ, USA

    Gautam Thaker

Authors
  1. Arvind S. Krishna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Balachandran Natarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aniruddha Gokhale
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Douglas C. Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nanbor Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gautam Thaker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arvind S. Krishna.

Additional information

Arvind S. Krishna is a PhD student in the Electrical Engineering and Computer Science Department at Vanderbilt University and a member of the Institute for Software Integrated Systems. He received his MA in management from the Brila Institute for Technology and Science (BITS), Pilani, India and his MS in computer science from University of California, Irvine. His research interests include patterns, real-time Java technologies for Real-Time Corba, model-integrated QA techniques, and tools for partial evaluation and specialization of middleware. He is a student member of the IEEE and ACM. Contact him at the Inst. for Software Integrated Systems, 2015 Terrace Pl., Nashville, TN 37203.

Balachandran Natarajan is a senior staff engineer at the Institute for Software Integrated Systems and a PhD student in electrical engineering and computer science at Vanderbilt University. His research focuses on applying patterns, optimization principles, and frameworks to build high-performance, dependable, and real-time distributed systems. He received his MS in computer science from Washington University. Contact him at the Inst. for Software Integrated Systems, 2015 Terrace Pl., Nashville, TN 37203.

Aniruddha Gokhale is an assistant professor in the Electrical Engineering and Computer Science Department at Vanderbilt University and a senior research scientist at the Institute for Software Integrated Systems. His research focuses on real-time component middleware optimizations, distributed systems and networks, model-driven software synthesis applied to component middleware-based distributed systems, and distributed resource management. He received his PhD in computer science from Washington University. Contact him at the Inst. for Software Integrated Systems, 2015 Terrace Pl., Nashville, TN 37203.

Douglas C. Schmidt is a professor in the Electrical Engineering and Computer Science Department at Vanderbilt University and a senior research scientist at the Institute for Software Integrated Systems. His research interests include patterns, optimization techniques, and empirical analyses of software frameworks and domain-specific modeling environments that facilitate the development of distributed real-time and embedded middleware and applications running over high-speed networks and embedded system interconnects. He received his PhD in information and computer science at the University of California, Irvine. Contact him at the Inst. for Software Integrated Systems, 2015 Terrace Pl., Nashville, TN 37203.

Nanbor Wang is a Research Scientist in the Distributed Technologies Group at the Tech-X Corporation in Boulder, Colorado. He received M.S. and Ph.D. degrees in Computer Science from Washington University in St. Louis, Missouri. While working for his degree, he also worked as a Research Associate in the Center of Distributed Object Computing in the Department of Computer Science where he conducted research on design, implementation and analysis of object-oriented and component-based techniques for development of distributed systems and management of extra-functional concerns. Dr. Wang’s work currently focuses on developing and applying middleware techniques, such as CORBA and Grid Computing, for enabling distributed and parallel scientific applications, such as, distributed data analysis, remote visualization and collaboration, and, work-flow management for large-scale scientific applications.

Gautam H. Thaker was born in Amdavad, India, in 1955. He holds a BSEE (75) and MSEE (77) from Clemson University, Clemson, SC. He spent the 85-86 academic year at M.I.T. as a visiting researcher. His research interests include analysis, design, construction and validation of real-time, command and control systems. In particular he has focused on interactions between operating systems, networking protocols, and middleware technologies.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Krishna, A.S., Natarajan, B., Gokhale, A. et al. CCMPerf: A Benchmarking Tool for CORBA Component Model Implementations. Real-Time Syst 29, 281–308 (2005). https://doi.org/10.1007/s11241-005-6889-6

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11241-005-6889-6

Keywords

Search

Navigation