skip to main content
10.1145/2652524.2652562acmconferencesArticle/Chapter ViewAbstractPublication PagesesemConference Proceedingsconference-collections
research-article

Using qualitative metasummary to synthesize empirical findings in literature reviews

Published: 18 September 2014 Publication History

Abstract

Context- A common problem in Systematic Reviews in software engineering is that they provide very limited syntheses. Goal- In the search for alternatives of effective methods for synthesizing empirical evidence, in this paper, we explore the use of the Qualitative Metasummary method, which is a quantitatively oriented aggregation of mixed research findings. Method - We describe the use of qualitative metasummary through an example using 15 studies addressing antecedents of performance of software development teams. Qualitative metasummary includes extraction and grouping of findings, and calculation of frequency and intensity effect sizes. Results -- The instance described in this paper produced a 10-factor model that effectively summarizes the current empirical knowledge on performance of software development teams. Then, we assessed the method in terms of ease of use, usefulness and reliability of results. Conclusion -- The Qualitative Metasummary method offers rich indexes of experiences and events under investigation, focusing on the effects of a variable over other, which is consistent with the central interest of systematic reviews. However, its main limitations are (i) challenging comparability/integratability between primary studies, (ii) loss of detailed contextual information, (iii) and the great deal of effort demanded to synthesize larger sets of papers.

References

[1]
Cruzes, Daniela and Dyba, Tore. Research synthesis in software engineering: A tertiary study. Information and Software Technology (2011), 440--455.
[2]
da Silva, Fabio, Cruz, Shirley, Gouveia, Tatiana, and Capretz, Luiz Fernando. Using meta-ethnography to synthesize research: A worked example of the relations between personality. In Empirical Software Engineering and Measurement (Baltimore, MD 2013).
[3]
da Silva, Fabio Q. B., Santos, André L. M., Soares, Sérgio, França, A. César C., Monteiro, Cleviton V. F., and Maciel, Felipe F. Six years of systematic literature reviews in software engineering: An updated tertiary study. Information and Software Technology, 53, 9 (2011), 899--913.
[4]
Kitchenham, Barbara Ann, Charters, Stuart, Budgen, David et al. Guidelines for performing Systematic Literature Reviews in Software Engineering. Keele, Staffordshire, UK, 2007.
[5]
Maxwell, JA. Maxwell JA. Understanding and validity in qualitative research. Harvard Educational Review, 1992.
[6]
M, Sandelowsk, J, Barroso, and C, Voils. Using qualitative metasummary to synthesise qualitative and quantitative descriptive findings. Research in Nursing & Health, 30 (2007), 99--111.
[7]
Onwuegbuzie, AJ and C, Teddlie. A framework for analyzing data in mixed methods research. In Tashakkori A, Teddlie C, editors, ed., Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage, 2003.
[8]
Sandelowski M, Barroso J 2003 Jul and 13(6):781--820. Writing the proposal for a qualitative research methodology project. Qual Health Res., 6, 13 (Jul 2003), 781--820.
[9]
Sandelowski, M and Barroso, J. Classifying the findings in qualitative studies. s. 2003 Sep; 13(7):905--23. Qual Health Res, 905, 23 (Sep 2003), 13.
[10]
Sandelowski, M and Barroso, J. Handbook for Synthesizing Qualitative Research. Springer, New York, 2006.
[11]
Sandelowski, M, Barroso, J, and Voils, C. Using Qualitative Metasummary to Synthesize Qualitative and Quantitative Descriptive Findings. Res Nurs Health. 2007 February, 1, 30 (fEB 2007), 99--111.
[12]
Shirley Cruz; Fabio Q. B. da Silva; Cleviton V. F. Monteiro; Pedro C. F. Santos; Isabella R. M. dos Santos. Personality in software engineering: Preliminary findings from a systematic literature review. In: 15th International Conference on Evaluation and Assessment in Software Engineering, 2011, Durham. Proceedings of the 15th International Conference on Evaluation and Assessment in Software Engineering. UK: Institute of Engineering and Technology, 2011. p. 1--10.
[13]
A. César C. França; Tatiana B. Gouveia; Pedro C. F. Santos; Célio A. Santana; Fabio Q. B. da Silva. Motivation in Software Engineering: a Systematic Review Update. In: 15th International Conference on Evaluation and Assessment in Software Engineering, 2011, Durham. Proceedings of the 15th International Conference on Evaluation and Assessment in Software Engineering, 2011.
[14]
Fabio Q. B. da Silva; Catarina Costa; A. César C. França; Rafael Prikladnicki. Challenges and Solutions in Distributed Software Development Project Management: a Systematic Literature Review. In: 5th IEEE International Conference on Global Software Engineering, 2010, Princeton. Proceedings of the 5th IEEE International Conference on Global Software Engineering, 2010. p. 87--96.
[15]
Marks, M., & Mathieu, J. (2001). A temporally based framework and taxonomy of team processes. Academy of Management Review, 26(3), 356--376.
[16]
{PS01} Demirors, Elif, Sarmagik, Gamze, and Demirors, Onur. The Role of Teamwork in Software Development Microsoft Case Study. EUROMICRO (1997), page 129--133.
[17]
{PS02} Rajendran, Mazhil. Analysis of team effectiveness in software development teams working on hardware and software environments using Belbin Self-perception Inventory. Journalof Management Development, 24, 8 (2005), 738--753.
[18]
{PS03} Pieterse, Vreda, Kourie, Derrick, and Sonnekus, Inge. Software Engineering Team Diversity and Performance. SAICSIT (2006), 180--186.
[19]
{PS04} Gorla, Narasimhaiah and Lam, Yan Wah. Who Should Work with Whom?.CommunicationsoftheACM, 47, 6 (2004), 79--82.
[20]
{PS05} Webster, Jane. Group Spontaneity. SIGCPR, 259--267.
[21]
{PS06} Jones, Mary and Harrison, Allison. IS project team performance: An empirical assessment. Information&Management, 31 (1996), 57--65.
[22]
{PS07} Yanga, Heng-Li and Tang, Jih-Hsin. Team structure and team performance in IS development: a social network perspective. Information& Management, 41 (2004), 335--349
[23]
{PS08} Hoeg, Martin andGemuenden, Hans Georg. Teamwork Quality and the Success of Innovative Projects: A Theoretical Concept and Empirical Evidence. Organization Science, 12, 4 (2001), 435--449
[24]
{PS09} Bradley, John H. and Hebert, Frederic J. The effect of personality type on team performance. Journalof Management Development, 16, 5 (1997)
[25]
{PS10} Lakhanpal, B. Understanding the factors influencing the performance of software development groups: an exploratory group-level analysis. Information and Software Technology (1993), 486--473.
[26]
{PS11} Hasan, Bassam and Ali, Jafar. An Empirical Examination of Factors affecting group effectiveness in information systems projects. Decision Sciences Journal of Innovative Education, 5, 2 (2007), 229--243
[27]
{PS12} Baddoo, Nathan, Hall, Tracy, and Jagielska, Dorota. Software Developer Motivation in a High Maturity Company: a Case Study. software process improvement and practice, 11 (2006), 219--228.
[28]
{PS13} Hoegl, Martin, Parboteeah, K. Praveen, and Gemuenden, Hans Georg. When teamwork really matters: task innovativeness as a moderator of the teamwork--performance relationship in software development projects. J. Eng. Technol. Manage, 20 (2003), 281--302.
[29]
{PS14} Bahli, Bouchaib and Büyükkurt, MeralDemirbaǧ. Group Performance in Information Systems Project Groups: An Empirical Study. Journal of Information Technology Education, 4 (2005), 97--113.
[30]
{PS15} Hoegl, Martin and Parboteeah, Praveen. Creativity in innovative projects: How teamwork matters. J. Eng. Technol. Manage., 24 (2007), 148--166

Cited By

View all
  • (2024)Guidance for engagement in health guideline development: A scoping reviewCampbell Systematic Reviews10.1002/cl2.7000620:4Online publication date: 25-Nov-2024
  • (2023)A Rapid Review of Sociocultural Dimensions in Nigeria’s Solid Waste Management ApproachInternational Journal of Environmental Research and Public Health10.3390/ijerph2013624520:13(6245)Online publication date: 28-Jun-2023
  • (2023)A Meta-Summary of Challenges in Building Products with ML Components – Collecting Experiences from 4758+ Practitioners2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)10.1109/CAIN58948.2023.00034(171-183)Online publication date: May-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ESEM '14: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement
September 2014
461 pages
ISBN:9781450327749
DOI:10.1145/2652524
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 September 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. qualitative metasummary
  2. research synthesis
  3. systematic review
  4. team performance

Qualifiers

  • Research-article

Funding Sources

Conference

ESEM '14
Sponsor:

Acceptance Rates

ESEM '14 Paper Acceptance Rate 23 of 123 submissions, 19%;
Overall Acceptance Rate 130 of 594 submissions, 22%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)23
  • Downloads (Last 6 weeks)0
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Guidance for engagement in health guideline development: A scoping reviewCampbell Systematic Reviews10.1002/cl2.7000620:4Online publication date: 25-Nov-2024
  • (2023)A Rapid Review of Sociocultural Dimensions in Nigeria’s Solid Waste Management ApproachInternational Journal of Environmental Research and Public Health10.3390/ijerph2013624520:13(6245)Online publication date: 28-Jun-2023
  • (2023)A Meta-Summary of Challenges in Building Products with ML Components – Collecting Experiences from 4758+ Practitioners2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)10.1109/CAIN58948.2023.00034(171-183)Online publication date: May-2023
  • (2021)Gamification in Software Engineering: A literature Review2021 IEEE/ACM 13th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE)10.1109/CHASE52884.2021.00020(105-108)Online publication date: May-2021
  • (2016)What Aspects of Context Should Be Described in Case Studies About Software Teams? Preliminary Results from a Mapping StudyProduct-Focused Software Process Improvement10.1007/978-3-319-49094-6_61(723-730)Online publication date: 6-Nov-2016
  • (2015)What Do We Know about High Performance Teams in Software Engineering? Results from a Systematic Literature ReviewProceedings of the 2015 41st Euromicro Conference on Software Engineering and Advanced Applications10.1109/SEAA.2015.24(183-190)Online publication date: 26-Aug-2015

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media