Abstract
Middleware architectures play a crucial role in determining the overall quality of many distributed applications. Systematic evaluation methods for middleware architectures are therefore important to thoroughly assess the impact of design decisions on quality goals. This paper presents MEMS, a scenario-based evaluation approach. MEMS provides a principled way of evaluating middleware architectures by leveraging generic qualitative and quantitative evaluation techniques such as prototyping, testing, rating, and analysis. It measures middleware architectures by rating multiple quality attributes, and the outputs aid the determination of the suitability of alternative middleware architectures to meet an application’s quality goals. MEMS also benefits middleware development by uncovering potential problems at early stage, making it cheaper and quicker to fix design problems. The paper describes a case study to evaluate the security architecture of grid middleware architectures for managing secure conversations and access control. The results demonstrate the practical utility of MEMS for evaluating middleware architectures for multiple quality attributes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ali Babar, M., Gorton, I.: Comparison of Scenario-Based Software Architecture Evaluation Methods. In: 11th Asia-Pacific Software Engineering Conference, pp. 600–607 (2004)
Barbacci, M., Klein, M.H., Longstaff, T.A., Weinstock, C.B.: Quality Attributes, Technical report, CMU/SEI-95-TR-021 (1995), http://www.sei.cmu.edu/publications/documents/95.reports/95.tr.021.html
Bass, L., Clements, P., Kazman, R.: Software Architecture in Practice, 2nd edn. SEI Series. Addison-Wesley, Reading (2003)
Bass, L., John, B.E., Kates, J.: Achieving usability through software architecture, Technical Report, CMU/SEI-2001-TR-005, http://www.sei.cmu.edu/publications/documents/01.reports/01tr005.html
Boehm, B., In, H.: Identifying Quality-Requirement Conflicts. IEEE Software 13(2), 25–35 (1996)
Cecchet, E., Marguerite, J., Zwaenepoel, W.: Performance and scalability of EJB applications. In: ACM Conference on Object-Oriented Programming, Systems, Languages and Applications (Oopsla 2002) (2002)
Cronbach, L.J.: Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–333 (1951)
Dobrica, L., Niemela, E.: A Survey on Software Architecture Analysis Methods. IEEE Transactions on Software Engineering 28(7) (2002)
Foster, I.: Service-oriented science. Science 308(5723), 814–817 (2005)
Foster, I., Kesselman, C., Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. In: Sakellariou, R., Keane, J.A., Gurd, J.R., Freeman, L. (eds.) Euro-Par 2001. LNCS, vol. 2150, p. 1. Springer, Heidelberg (2001)
Gorton, I., Liu, A., Brebner, P.: Rigorous Evaluation of COTS Middleware Technology. IEEE Computer 36(3), 50–55 (2003)
Globus Toolkit, http://www.globus.org/
Gwet, K.: Handbook of Inter-Rater Reliability, STATAXIS Publishing Company, ISBN, 0970806205
Harmer, T., Stell, A., McBride, D.: UK Engineering Task Force Globus Toolkit Version 4 Middleware Evaluation, UK Technical Report UKeS-2005-03 (2005)
Lawlor, B., Vu, L.: A Survey of Techniques for Security Architecture Analysis, Science and Technical Report, DSTO-TR-1438 (2003)
Nord, R.L., Tomayko, J.E.: Software Architecture-Centric Methods and Agile Development. IEEE Software 23(2), 47–53 (2006)
Liu, A., Gorton, I.: Accelerating COTS Middleware Acquisition: The i-Mate Process. IEEE Software 20(2), 72–79 (2003)
Liu, Y., Gorton, I., Abanmi, S., Hoang, C.: A secure configuration manager for adaptive server framework, NICTA Summer Scholarship Project Proposal (November 2005)
Jogalekar, P., Woodside, M.: Evaluating the Scalability of Distributed Systems. IEEE Transactions on Parallel and Distributed Systems 11(6), 589–603 (2000)
Kanoun, K., Kaaniche, M., Laprie, J.C.: Qualitative and Quantitative Reliability Assessment. IEEE Software, 77–87 (1997)
Kontio, J.: A Case Study in Applying a Systematic Method for COTS Selection. In: Proc. 18th Int’l. Conf. Software Eng., IEEE CS Press, Los Alamitos (1996)
Nord, R.L., Tomayko, J.E.: Software Architecture-Centric Methods and Agile Development. IEEE Software 23(2), 47–53 (2006)
Smith, C.U., Willams, L.G.: Performance Engineering of CORBA-based Distributed Systems with SPE·ED. In: Computer Performance Evaluation Modelling Techniques and Tools. LNCS. Springer, Heidelberg (1998)
Sotomayor, B., Childers, L.: Globus Toolkit 4: Programming Java Services, written by Borja Sotomayor and Lisa Childers. Elsevier, Amsterdam
Tang, D., Kumar, D., Duvur, S., Torbjornsen, O.: Availability measurement and modeling for an application server. In: International Conference on Dependable Systems and Networks, pp. 669–678 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, Y., Gorton, I., Bass, L., Hoang, C., Abanmi, S. (2006). MEMS: A Method for Evaluating Middleware Architectures. In: Hofmeister, C., Crnkovic, I., Reussner, R. (eds) Quality of Software Architectures. QoSA 2006. Lecture Notes in Computer Science, vol 4214. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11921998_6
Download citation
DOI: https://doi.org/10.1007/11921998_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-48819-4
Online ISBN: 978-3-540-48820-0
eBook Packages: Computer ScienceComputer Science (R0)