research-article

Assembling multiple-case studies: potential, principles and practical considerations

Authors:

Aiko Yamashita,

Leon MoonenAuthors Info & Claims

EASE '14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering

Article No.: 36, Pages 1 - 10

https://doi.org/10.1145/2601248.2601286

Published: 13 May 2014 Publication History

Abstract

Case studies are a research method aimed at holistically analyzing a phenomenon in its context. Despite the fact that they cannot be used to answer the same precise research questions as, e.g., can be addressed by controlled experiments, case studies can cope much better with situations having several variables of interest, multiple sources of evidence, or rich contexts that cannot be controlled or isolated. As such, case studies are a promising instrument to study the complex phenomena at play in Software Engineering.

However, the use of case studies as research methodology entails certain challenges. We argue that one of the biggest challenges is the case selection bias when conducting multiple-case studies. In practice, cases are frequently selected based on their availability, without appropriate control over moderator factors. This hinders the level of comparability across cases, leading to internal validity issues.

In this paper, we discuss the notion of assembling cases as a plausible alternative to selecting cases to overcome the selection bias problem when conducting multiple-case studies. In addition, we present and discuss our experiences from applying this approach in a study designed to investigate the impact of software design on maintainability.

References

[1]

O. Alonso, D. E. Rose, and B. Stewart. Crowdsourcing for relevance evaluation. ACM SIGIR Forum, 42(2):9, 2008.

Digital Library

[2]

B. C. D. Anda, D. I. K. Sjøberg, and A. Mockus. Variability and Reproducibility in Software Engineering: A Study of Four Companies that Developed the Same System. IEEE Trans. Softw. Eng., 35(3):407--429, 2009.

Digital Library

[3]

D. E. Avison, F. Lau, M. D. Myers, and P. A. Nielsen. Action research. Comm. ACM, 42(1):94--97, 1999.

Digital Library

[4]

I. Benbasat, D. K. Goldstein, and M. Mead. The Case Research Strategy in Studies of Information Systems. MIS Quarterly, 11(3):369, 1987.

Digital Library

[5]

G. R. Bergersen and J.-E. Gustafsson. Programming Skill, Knowledge, and Working Memory Among Professional Software Developers from an Investment Theory Perspective. J. Individual Differences, 32(4):201--209, 2011.

[6]

J. W. Creswell and V. L. P. Clark. Designing and conducting mixed methods research. SAGE, 2007.

[7]

D. S. Cruzes, T. Dyba, P. Runeson, and M. Host. Case Studies Synthesis: Brief Experience and Challenges for the Future. In 2011 Int'l Symp. Empirical Softw. Eng. and Measurement, pages 343--346. IEEE, 2011.

Digital Library

[8]

K. M. Eisenhardt. Building Theories from Case Study Research. Academy of Management Review, 14(4), 1989.

[9]

B. Flyvberg. Five misunderstandings about case-study research. In Qualitative Research Practice. Sage, 2007.

[10]

M. Fowler. Refactoring: Improving the Design of Existing Code. Addison-Wesley, 1999.

Digital Library

[11]

A. L. George and A. Bennett. Case studies and theory development in the social sciences. MIT Press, 2005.

[12]

J. E. Hannay, D. I. K. Sjøberg, and T. Dybå. A Systematic Review of Theory Use in Software Engineering Experiments. IEEE Trans. Softw. Eng., 33(2):87--107, 2007.

Digital Library

[13]

L. M. Layman, L. A. Williams, and R. St. Amant. MimEc. In Int'l Ws. Cooperative and Human Aspects of Softw. Eng. (CHASE), pages 73--76. ACM Press, 2008.

Digital Library

[14]

A. Lijphart. Comparative Politics and the Comparative Method. The American Political Science Review, 65(3):pp. 682--693, 1971.

[15]

C. Ragin. The comparative method: Moving beyond qualitative and quantitative strategies. University of California Press, 1987.

[16]

P. Runeson, M. Höst, A. Rainer, and B. Regnell. Case Study Research in Software Engineering: Guidelines and Examples. Wiley, 2012.

Digital Library

[17]

O. Salo and P. Abrahamsson. Empirical Evaluation of Agile Software Development: The Controlled Case Study Approach. In Product-Focused Softw. Process Improvement, pages 408--423, 2004.

[18]

D. I. Sjoberg, A. Yamashita, B. C. Anda, A. Mockus, and T. Dyba. Quantifying the Effect of Code Smells on Maintenance Effort. IEEE Trans. Softw. Eng., 39(8):1144--1156, 2013.

Digital Library

[19]

A. Strauss and J. Corbin. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE, 1998.

[20]

W. M. K. Trochim. Outcome pattern matching and program theory. Evaluation and Program Planning, 12(4):355--366, 1989.

[21]

A. Yamashita and L. Moonen. Do code smells reflect important maintainability aspects? In Int'l Conf. Softw. Maintenance, pages 306--315. IEEE, 2012.

Digital Library

[22]

A. Yamashita and L. Moonen. Exploring the impact of inter-smell relations on software maintainability: An empirical study. In 2013 35th Int'l Conf. Softw. Eng. (ICSE), pages 682--691. IEEE, 2013.

Digital Library

[23]

A. Yamashita and L. Moonen. To what extent can maintenance problems be predicted by code smell detection? - An empirical study. Information and Softw. Technology, 55(12):2223--2242, 2013.

Digital Library

[24]

R. Yin. Case Study Research: Design and Methods (Applied Social Research Methods). SAGE, 2002.

Cited By

Lakhdhir S(2024)Creating Positive Social Experiences Through the Design of Custom WearablesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638190(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3638190
Venkatesh RSinghal THallur G(2021)VIDE-LEOInternational Journal of e-Collaboration10.4018/IJeC.29029418:1(1-19)Online publication date: 22-Oct-2021
https://doi.org/10.4018/IJeC.290294

Index Terms

Assembling multiple-case studies: potential, principles and practical considerations
1. Software and its engineering

Recommendations

How do professionals perceive legacy systems and software modernization?
ICSE 2014: Proceedings of the 36th International Conference on Software Engineering

Existing research in legacy system modernization has traditionally focused on technical challenges, and takes the standpoint that legacy systems are obsolete, yet crucial for an organization's operation. Nonetheless, it remains unclear whether ...
A comparative study of challenges in integrating Open Source Software and Inner Source Software

Context: Several large software-developing organizations have adopted Open Source Software development (OSSD) practices to develop in-house components that are subsequently integrated into products. This phenomenon is also known as ''Inner Source''. ...
Mismatch handling for COTS selection: a case study

Using Commercial Off-The-Shelf (COTS) products to build software systems requires the evaluation of existing COTS products, the selection of the COTS that best fit system requirements, and the integration of the selected COTS into the system. During ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EASE '14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering

May 2014

486 pages

ISBN:9781450324762

DOI:10.1145/2601248

General Chair:
Martin Shepperd
Brunel University, UK
,
Program Chairs:
Tracy Hall
Brunel University
,
Ingunn Myrtveit
The Norwegian Business School

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Brunel University: Brunel University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

EASE '14

Sponsor:

Brunel University

EASE '14: 18th International Conference on Evaluation and Assessment in Software Engineering

May 13 - 14, 2014

England, London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
251
Total Downloads

Downloads (Last 12 months)35
Downloads (Last 6 weeks)5

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lakhdhir S(2024)Creating Positive Social Experiences Through the Design of Custom WearablesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638190(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3638190
Venkatesh RSinghal THallur G(2021)VIDE-LEOInternational Journal of e-Collaboration10.4018/IJeC.29029418:1(1-19)Online publication date: 22-Oct-2021
https://doi.org/10.4018/IJeC.290294

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten