skip to main content
research-article

Report from the 2nd international workshop on replication in empirical software engineering research (RESER 2011)

Published: 27 January 2012 Publication History

Abstract

The RESER workshop provides a venue in which empirical software engineering researchers can discuss the theoretical foundations and methods of replication, as well as present the results of specific replicated studies. In 2011, the workshop co-located with the International Symposium on Empirical Software Engineering and Measurement (ESEM) in Banff, Alberta, Canada. In addition to several outstanding paper sessions, highlights of the 2011 workshop included a keynote address by Dr. Victor R. Basili, in which he addressed the question, "What's so hard about replication of software engineering experiments?" The workshop also featured a joint replication panel session discussing the first cooperative joint replication ever conducted in empirical software engineering research and a planning session for next year's joint replication project addressing Conway's Law.

References

[1]
V. R. Basili. http://www.cs.umd.edu/~basili/, 2011. Accessed, October 2011.
[2]
A. Brooks, J. Daly, J. Miller, M. Roper, and M. Wood. Replication of experimental results in software engineering. Technical report, International Software Engineering Research Network, 1996.
[3]
A. Brooks, M. Roper, M. Wood, J. Daly, and J. Miller. Replication's role in software engineering. In F. Shull, J. Singer, and D. I. K. S. berg, editors, Guide to Advanced Empirical Software Engineering, pages 365--379. Springer, 2008.
[4]
S. H. Burton, P. M. Bodily, R. G. Morris, C. D. Knutson, and J. L. Krein. Design team perception of development team composition: Implications for Conway's Law. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Banff, AB, Canada, September 2011. IEEE Computer Society.
[5]
J. C. Carver. Towards reporting guidelines for experimental replications: A proposal. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Cape Town, South Africa, May 2010.
[6]
F. Q. B. da Silva, M. Suassuna, R. F. Lopes, T. B. Gouveia, A. C. A. França, J. P. N. de Oliveira, L. F. M. de Oliveira, and A. L. M. Santos. Replication of empirical studies in software engineering: Preliminary findings from a systematic mapping study. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Banff, AB, Canada, September 2011. IEEE Computer Society.
[7]
O. Dieste, E. Fernandez, R. Garcia, and N. Juristo. Hidden evidence behind useless replications. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Cape Town, South Africa, May 2010.
[8]
Freie Universität Berlin. Replication of 'PatMain'. http://www.inf.fu-berlin.de/w/SE/PatmainReplicationInfo, 2011. Accessed, October 2011.
[9]
C. Ghezzi. Reflections on 40+ years of software engineering research and beyond: An insider's view. Keynote address at the International Conference on Software Engineering, Vancouver, BC, Canada, May 2009.
[10]
A. Jedlitschka and D. Pfahl. Reporting guidelines for controlled experiments in software engineering. In Proceedings of the International Symposium on Empirical Software Engineering, pages 95--104, Noosa Heads, Australia, November 2005. IEEE Computer Society.
[11]
N. Juristo and S. Vegas. Design patterns in software maintenance: An experiment replication at UPM. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Banff, AB, Canada, September 2011. IEEE Computer Society.
[12]
C. D. Knutson, J. L. Krein, L. Prechelt, and N. Juristo. Report from the 1st international workshop on replication in empirical software engineering research (RESER 2010). SIGSOFT Software Engineering Notes, 35(5):42--44, 2010.
[13]
C. Kozak and M. Squire. A secondary data archive for code-level Debian metrics. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Banff, AB, Canada, September 2011. IEEE Computer Society.
[14]
J. L. Krein, L. J. Pratt, A. B. Swenson, A. C. MacLean, C. D. Knutson, and D. L. Eggett. Design patterns in software maintenance: An experiment replication at Brigham Young University. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Banff, AB, Canada, September 2011. IEEE Computer Society.
[15]
A. Nanthaamornphong and J. C. Carver. Design patterns in software maintenance: An experiment replication at University of Alabama. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Banff, AB, Canada, September 2011. IEEE Computer Society.
[16]
L. Prechelt and M. Liesenberg. Design patterns in software maintenance: An experiment replication at Freie Universität Berlin. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Banff, AB, Canada, September 2011. IEEE Computer Society.
[17]
L. Prechelt, B. Unger, W. F. Tichy, P. Brössler, and L. G. Votta. A controlled experiment in maintenance comparing design patterns to simpler solutions. IEEE Transactions on Software Engineering, 27(12):1134--1144, 2001.
[18]
P. Runeson, A. Stefik, A. Andrews, S. Grönblom, I. Porres, and S. Siebert. A comparative analysis of three replicated experiments comparing inspection and unit testing. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Banff, AB, Canada, September 2011. IEEE Computer Society.
[19]
F. Shull, V. Basili, J. Carver, J. Maldonado, G. Travassos, M. Mendon?ca, and S. Fabbri. Replicating software engineering experiments: Addressing the tacit knowledge problem. In Proceedings of the International Symposium on Empirical Software Engineering, pages 7--16, Nara, Japan, October 2002. IEEE Computer Society.
[20]
F. Shull, J. Carver, S. Vegas, and N. Juristo. The role of replications in empirical software engineering. Empirical Software Engineering, 13(2):211--218, 2008.
[21]
M. Vokáč, W. F. Tichy, D. I. K. Sjoberg, E. Arisholm, and M. Aldrin. A controlled experiment comparing the maintainability of programs designed with and without design patterns: A replication in a real programming environment. Empirical Software Engineering, 9(3):149--195, 2004.
[22]
E. J. Weyuker, R. M. Bell, and T. J. Ostrand. Replicate, replicate, replicate. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Banff, AB, Canada, September 2011. IEEE Computer Society.

Cited By

View all
  • (2016)A Multi-Site Joint Replication of a Design Patterns Experiment Using Moderator Variables to Generalize across ContextsIEEE Transactions on Software Engineering10.1109/TSE.2015.248862542:4(302-321)Online publication date: 1-Apr-2016
  • (2015)Investigations about replication of empirical studies in software engineeringInformation and Software Technology10.1016/j.infsof.2015.02.00164:C(76-101)Online publication date: 1-Aug-2015
  • (2014)Can plans and situated actions be replicated?Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing10.1145/2531602.2531627(603-614)Online publication date: 15-Feb-2014

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGSOFT Software Engineering Notes
ACM SIGSOFT Software Engineering Notes  Volume 37, Issue 1
January 2012
115 pages
ISSN:0163-5948
DOI:10.1145/2088883
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 January 2012
Published in SIGSOFT Volume 37, Issue 1

Check for updates

Author Tags

  1. experimentation
  2. methods
  3. replication
  4. reporting
  5. software engineering
  6. validation
  7. validity

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2016)A Multi-Site Joint Replication of a Design Patterns Experiment Using Moderator Variables to Generalize across ContextsIEEE Transactions on Software Engineering10.1109/TSE.2015.248862542:4(302-321)Online publication date: 1-Apr-2016
  • (2015)Investigations about replication of empirical studies in software engineeringInformation and Software Technology10.1016/j.infsof.2015.02.00164:C(76-101)Online publication date: 1-Aug-2015
  • (2014)Can plans and situated actions be replicated?Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing10.1145/2531602.2531627(603-614)Online publication date: 15-Feb-2014

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media