research-article

Evidence in software architecture, a systematic literature review

Authors:

Muhammad Usman,

Naveed IkramAuthors Info & Claims

EASE '13: Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering

Pages 97 - 106

https://doi.org/10.1145/2460999.2461014

Published: 14 April 2013 Publication History

Abstract

Context: Software architecture (SA) plays pivotal role in development and maintenance of large software systems. Architectural decisions impact all subsequent phases in software development life cycle. Structure of large software systems has been in discussion since early 70s but software architecture started emerging as separate discipline since mid-90s. Objective: Researchers have been publishing empirical studies (i.e. case studies, experiments, experience reports) in SA discipline. Empirical literature has been aggregated in other disciplines of software engineering but no such effort has been made in SA. Objective of this study is to aggregate and synthesize the empirical evidence from literature of software architecture to report the trends, patterns and knowledge gaps. Method: To synthesize the empirical work in SA, we conducted a systematic literature review (SLR). This study reports the results of SLR based on 247 included primary studies. Results: Based on the percentage of 247 primary studies we identified that SA Evaluation (28%), non- functional requirements related work (22%) and SA Design (12%) are relatively mature sub areas of SA. Most of the empirical work (59%) in SA uses case study research method. Experiments (20%) and experience reports (14%) are also employed in empirical SA work. Conclusion: SA discipline is maturing as few sub areas of SA are mature and some areas are new and being developed. We have also identified few emerging trends in SA i.e. service oriented architecture, Product line architecture, Aspect oriented architecture, and model driven architecture.

References

[1]

D. E. Perry and A. L. Wolf, "Foundations for the study of software architecture," ACM SIGSOFT Software Engineering Notes, vol. 17, no. 4, pp. 40--52, 1992.

Digital Library

[2]

P. Kruchten, H. Obbink, and J. Stafford, "The past, present, and future for software architecture," Software, IEEE, vol. 23, no. 2, pp. 22--30, 2006.

Digital Library

[3]

D. Garlan, "Research directions in software architecture," ACM Computing Surveys (CSUR), vol. 27, no. 2, pp. 257--261, 1995.

Digital Library

[4]

B. J. Oates, "Widening the scope of evidence gathering in software engineering," in Software Technology and Engineering Practice, 2003. Eleventh Annual International Workshop on, 2003, pp. 59--64.

Digital Library

[5]

T. Dyba, B. A. Kitchenham, and M. Jorgensen, "Evidence-based software engineering for practitioners," Software, IEEE, vol. 22, no. 1, pp. 58--65, 2005.

Digital Library

[6]

C. Gacek, A. Abd-Allah, B. Clark, and B. Boehm, "On the definition of software system architecture," in Proceedings of the First International Workshop on Architectures for Software Systems, 1995, pp. 85--94.

[7]

D. E. Perry, "State of the Art: Software Architecture," in International Conference on Software Engineering, 1997, vol. 19, pp. 590--591.

Digital Library

[8]

Y. Chen, X. Li, L. Yi, D. Liu, L. Tang, and H. Yang, "A ten-year survey of software architecture," in Software Engineering and Service Sciences (ICSESS), 2010 IEEE International Conference on, 2010, pp. 729--733.

[9]

M. Shaw, "The coming-of-age of software architecture research," in Proceedings of the 23rd international conference on Software engineering, 2001, p. 656.

Digital Library

[10]

M. Shaw and P. Clements, "The golden age of software architecture," Software, IEEE, vol. 23, no. 2, pp. 31--39, 2006.

Digital Library

[11]

D. Falessi, M. A. Babar, G. Cantone, and P. Kruchten, "Applying empirical software engineering to software architecture: challenges and lessons learned," Empirical Software Engineering, vol. 15, no. 3, pp. 250--276, 2010.

Digital Library

[12]

H. P. Breivold, I. Crnkovic, and M. Larsson, "A systematic review of software architecture evolution research," Information and Software Technology, 2011.

Digital Library

[13]

R. Farenhorst and R. C. de Boer, "Knowledge management in software architecture: State of the art," Software Architecture Knowledge Management, vol. 1, p. 21, 2009.

[14]

B. J. Williams and J. C. Carver, "Characterizing software architecture changes: A systematic review," Information and Software Technology, vol. 52, no. 1, pp. 31--51, 2010.

Digital Library

[15]

E. Y. Nakagawa, D. Feitosa, and K. R. Felizardo, "Using systematic mapping to explore software architecture knowledge," in Proceedings of the 2010 ICSE Workshop on Sharing and Reusing Architectural Knowledge, 2010, pp. 29--36.

Digital Library

[16]

B. Kitchenham and S. Charters, "Guidelines for performing systematic literature reviews in software engineering," Engineering, vol. 2, no. EBSE 2007--001, 2007.

[17]

N. Qureshi, N. Ikram, M. Bano, and M. Usman, "Empirical Evidence in Software Architecture: A Systematic Literature Review Protocol," in ICSEA 2011, The Sixth International Conference on Software Engineering Advances, 2011, pp. 534--538.

[18]

B. A. Kitchenham, O. P. Brereton, D. Budgen, and Z. Li, "An Evaluation of Quality Checklist Proposals-A participant-observer case study," in 13th International Conference on Evaluation and Assessment in Software Engineering, 2009.

Digital Library

[19]

B. Kitchenham, D. I. K. Sjøberg, O. P. Brereton, D. Budgen, T. Dyb\aa, M. Höst, D. Pfahl, and P. Runeson, "Can we evaluate the quality of software engineering experiments?," in Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, 2010, p. 2.

Digital Library

[20]

M. Host and P. Runeson, "Checklists for software engineering case study research," in Empirical Software Engineering and Measurement, 2007. ESEM 2007. First International Symposium on, 2007, pp. 479--481.

Digital Library

[21]

D. Budgen and C. Zhang, "Preliminary reporting guidelines for experience papers," in Proceedings of EASE, 2009, vol. 2009, pp. 1--10.

Digital Library

[22]

T. Dyb\aa and T. Dingsøyr, "Empirical studies of agile software development: A systematic review," Information and software technology, vol. 50, no. 9, pp. 833--859, 2008.

Digital Library

[23]

R. Wieringa, N. Maiden, N. Mead, and C. Rolland, "Requirements engineering paper classification and evaluation criteria: a proposal and a discussion," Requirements Engineering, vol. 11, no. 1, pp. 102--107, 2006.

Digital Library

[24]

K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson, "Systematic mapping studies in software engineering," in 12th International Conference on Evaluation and Assessment in Software Engineering, 2008, pp. 71--80.

Digital Library

[25]

D. L. Parnas, "On the criteria to be used in decomposing systems into modules," Communications of the ACM, vol. 15, no. 12, pp. 1053--1058, 1972.

Digital Library

Cited By

Soliman MAlbonico MMalavolta IWortmann A(2025)Mining software repositories for software architecture — A systematic mapping studyInformation and Software Technology10.1016/j.infsof.2025.107677181(107677)Online publication date: May-2025
https://doi.org/10.1016/j.infsof.2025.107677
Galster MWeyns D(2023)Empirical research in software architecture — Perceptions of the communityJournal of Systems and Software10.1016/j.jss.2023.111684202(111684)Online publication date: Aug-2023
https://doi.org/10.1016/j.jss.2023.111684
Konersmann MKaplan AKuhn THeinrich RKoziolek AReussner RJurjens Jal-Doori MBoltz NEhl MFuchs DGroser KHahner SKeim JLohr MSaglam TSchulz SToberg J(2022)Evaluation Methods and Replicability of Software Architecture Research Objects2022 IEEE 19th International Conference on Software Architecture (ICSA)10.1109/ICSA53651.2022.00023(157-168)Online publication date: Mar-2022
https://doi.org/10.1109/ICSA53651.2022.00023
Show More Cited By

Index Terms

Evidence in software architecture, a systematic literature review
1. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Software architectures

Recommendations

A systematic literature review of traceability approaches between software architecture and source code
EASE '14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering

The links between the software architecture and the source code of a software system should be based on solid traceability mechanisms in order to effectively perform quality control and maintenance of the software system. There are several primary ...
Guidelines for snowballing in systematic literature studies and a replication in software engineering
EASE '14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering

Background: Systematic literature studies have become common in software engineering, and hence it is important to understand how to conduct them efficiently and reliably.

Objective: This paper presents guidelines for conducting literature reviews using ...
Systematic literature reviews in software engineering - A systematic literature review

Background: In 2004 the concept of evidence-based software engineering (EBSE) was introduced at the ICSE04 conference. Aims: This study assesses the impact of systematic literature reviews (SLRs) which are the recommended EBSE method for aggregating ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EASE '13: Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering

April 2013

268 pages

ISBN:9781450318488

DOI:10.1145/2460999

General Chair:
Fabio Q. B. da Silva
Universidade Federal de Pernambuco, Brazil
,
Program Chairs:
Natalia Juristo
Universidad Politécnica de Madrid, Spain
,
Guilherme Horta Travassos
Universidade Federal do Rio de Janeiro, Brazil

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Centro de Informatica - UFPE: Centro de Informatica - UFPE
SBC: Brazilian Computer Society
CNPq: Conselho Nacional de Desenvolvimento Cientifico e Tecn
CAPES: Brazilian Higher Education Funding Council

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 April 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

EASE '13

Sponsor:

Centro de Informatica - UFPE
SBC
CNPq
CAPES

EASE '13: 17th International Conference on Evaluation and Assessment in Software Engineering

April 14 - 16, 2013

Porto de Galinhas, Brazil

Acceptance Rates

EASE '13 Paper Acceptance Rate 31 of 94 submissions, 33%;

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
823
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)3

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Soliman MAlbonico MMalavolta IWortmann A(2025)Mining software repositories for software architecture — A systematic mapping studyInformation and Software Technology10.1016/j.infsof.2025.107677181(107677)Online publication date: May-2025
https://doi.org/10.1016/j.infsof.2025.107677
Galster MWeyns D(2023)Empirical research in software architecture — Perceptions of the communityJournal of Systems and Software10.1016/j.jss.2023.111684202(111684)Online publication date: Aug-2023
https://doi.org/10.1016/j.jss.2023.111684
Konersmann MKaplan AKuhn THeinrich RKoziolek AReussner RJurjens Jal-Doori MBoltz NEhl MFuchs DGroser KHahner SKeim JLohr MSaglam TSchulz SToberg J(2022)Evaluation Methods and Replicability of Software Architecture Research Objects2022 IEEE 19th International Conference on Software Architecture (ICSA)10.1109/ICSA53651.2022.00023(157-168)Online publication date: Mar-2022
https://doi.org/10.1109/ICSA53651.2022.00023
Campoverde‐Molina MLuján‐Mora SValverde L(2021)Systematic literature review on software architecture of educational websitesIET Software10.1049/sfw2.1202415:4(239-259)Online publication date: 21-Apr-2021
https://doi.org/10.1049/sfw2.12024
Martensson TStahl DMartini ABosch J(2019)Continuous Architecture: Towards the Goldilocks Zone and Away from Vicious Circles2019 IEEE International Conference on Software Architecture (ICSA)10.1109/ICSA.2019.00022(131-140)Online publication date: Mar-2019
https://doi.org/10.1109/ICSA.2019.00022
Anish PBalasubramaniam BSainani ACleland-Huang JDaneva MWieringa RGhaisas SDillon LVisser WWilliams L(2016)Probing for requirements knowledge to stimulate architectural thinkingProceedings of the 38th International Conference on Software Engineering10.1145/2884781.2884801(843-854)Online publication date: 14-May-2016
https://dl.acm.org/doi/10.1145/2884781.2884801
Anish P(2016)Towards an Approach to Stimulate Architectural Thinking during Requirements Engineering Phase2016 IEEE 24th International Requirements Engineering Conference (RE)10.1109/RE.2016.30(421-426)Online publication date: Sep-2016
https://doi.org/10.1109/RE.2016.30
Ambreen TIkram NUsman MNiazi M(2016)Empirical research in requirements engineering: trends and opportunitiesRequirements Engineering10.1007/s00766-016-0258-223:1(63-95)Online publication date: 29-Jul-2016
https://doi.org/10.1007/s00766-016-0258-2
Anish PDaneva MCleland-Huang JWieringa RGhaisas S(2015)What you ask is what you get: Understanding architecturally significant functional requirements2015 IEEE 23rd International Requirements Engineering Conference (RE)10.1109/RE.2015.7320411(86-95)Online publication date: Aug-2015
https://doi.org/10.1109/RE.2015.7320411
Gonzalez-Huerta JInsfran EAbrahão SScanniello G(2015)Validating a model-driven software architecture evaluation and improvement method: A family of experimentsInformation and Software Technology10.1016/j.infsof.2014.05.01857(405-429)Online publication date: Jan-2015
https://doi.org/10.1016/j.infsof.2014.05.018
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten