skip to main content
review-article

Report from the 3rd International Workshop on Replication in Empirical Software Engineering Research (RESER 2013)

Published: 11 February 2014 Publication History

Abstract

The RESER workshop provides a venue in which empirical software engineering researchers can discuss the theoretical foundations and methods of replication, as well as present the results of specific replicated studies. In 2013, the workshop co-located with the International Symposium on Empirical Software Engineering and Measurement (ESEM) in Baltimore, Maryland. In addition to several outstanding paper sessions, highlights of the 2013 workshop included a keynote address by Dr. Mel Conway, in which he presented a "45-Year Retrospective on Conway's Law and the Sociology of System Design." The workshop also featured a joint replication session exploring the results and methodologies of four research teams, who each studied some aspect of Conway's Law.

References

[1]
S. E. Bailey, S. S. Godbole, C. D. Knutson, and J. L. Krein. A decade of Conway's Law: A literature review from 2003-2012. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 1--14, Baltimore, Maryland, October 2013. IEEE Computer Society.
[2]
S. Betz, D. Šmite, S. Fricker, A. Moss, W. Afzal, M. Svahnberg, C. Wohlin, J. Börstler, and T. Gorschek. An evolutionary perspective on socio-technical congruence: The rubber band effect. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 15--24, Baltimore, Maryland, October 2013. IEEE Computer Society.
[3]
K. L. Blatter, T. J. Gledhill, J. L. Krein, and C. D. Knutson. Impact of communication structure on system design: Towards a controlled test of Conway's Law. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 25--33, Baltimore, Maryland, October 2013. IEEE Computer Society.
[4]
A. Brooks, J. Chambers, C. N. Lee, and F. Mead. A partial replication with a sample size of one: A smoke test for empirical software engineering. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 56--65, Baltimore, Maryland, October 2013. IEEE Computer Society.
[5]
A. Brooks, J. Daly, J. Miller, M. Roper, and M. Wood. Replication of experimental results in software engineering. Technical report, International Software Engineering Research Network, 1996.
[6]
A. Brooks, M. Roper, M. Wood, J. Daly, and J. Miller. Replication's role in software engineering. In F. Shull, J. Singer, and D. I. K. S. berg, editors, Guide to Advanced Empirical Software Engineering, pages 365--379. Springer, 2008.
[7]
J. C. Carver. Towards reporting guidelines for experimental replications: A proposal. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Cape Town, South Africa, May 2010.
[8]
M. Conway. http://melconway.com/keynote/, 2013. Accessed, November 2013.
[9]
C. V. C. de Magalhäes and F. Q. B. da Silva. Towards a taxonomy of replications in empirical software engineering research: A research proposal. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 50--55, Baltimore, Maryland, October 2013. IEEE Computer Society.
[10]
A. M. de Santana, F. Q. B. da Silva, R. C. G. de Miranda, A. A. Mascaro, T. B. Gouveia, C. V. F. Monteiro, and A. L. M. Santos. Relationships between communication structure and software architecture: An empirical investigation of the Conway's Law at the Federal University of Pernambuco. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 34--42, Baltimore, Maryland, October 2013. IEEE Computer Society.
[11]
O. Dieste, E. Fernandez, R. Garcia, and N. Juristo. Hidden evidence behind useless replications. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, Cape Town, South Africa, May 2010.
[12]
K. R. Felizardo, S. R. S. Souza, and J. C. Maldonado. The use of visual text mining to support the study selection activity in systematic literature reviews: A replication study. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 91--100, Baltimore, Maryland, October 2013. IEEE Computer Society.
[13]
C. Ghezzi. Reflections on 40+ years of software engineering research and beyond: An insider's view. Keynote address at the International Conference on Software Engineering, Vancouver, BC, Canada, May 2009.
[14]
A. Jedlitschka and D. Pfahl. Reporting guidelines for controlled experiments in software engineering. In Proceedings of the International Symposium on Empirical Software Engineering, pages 95--104, Noosa Heads, Australia, November 2005. IEEE Computer Society.
[15]
B. Penzenstadler, J. Eckhardt, and D. M. Fernández. Two replication studies for evaluating artefact models in RE: Results and lessons learnt. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 66--75, Baltimore, Maryland, October 2013. IEEE Computer Society.
[16]
A. S. Sayyad, K. Goseva-Popstojanova, T. Menzies, and H. Ammar. On parameter tuning in search based software engineering: A replicated empirical study. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 84--90, Baltimore, Maryland, October 2013. IEEE Computer Society.
[17]
F. Shull, V. Basili, J. Carver, J. Maldonado, G. Travassos, M. Mendon?ca, and S. Fabbri. Replicating software engineering experiments: Addressing the tacit knowledge problem. In Proceedings of the International Symposium on Empirical Software Engineering, pages 7--16, Nara, Japan, October 2002. IEEE Computer Society.
[18]
F. Shull, J. Carver, S. Vegas, and N. Juristo. The role of replications in empirical software engineering. Empirical Software Engineering, 13(2):211--218, 2008.
[19]
M. Squire. A replicable infrastructure for empirical studies of email archives. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 43--49, Baltimore, Maryland, October 2013. IEEE Computer Society.
[20]
M. F. Tennyson. A replicated comparative study of source code authorship attribution. In Proceedings of the International Workshop on Replication in Empirical Software Engineering Research, pages 76--83, Baltimore, Maryland, October 2013. IEEE Computer Society.

Cited By

View all
  • (2015)Six strategies for generalizing software engineering theoriesScience of Computer Programming10.1016/j.scico.2014.11.013101(136-152)Online publication date: Apr-2015
  • (2015)Investigations about replication of empirical studies in software engineeringInformation and Software Technology10.1016/j.infsof.2015.02.00164:C(76-101)Online publication date: 1-Aug-2015

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGSOFT Software Engineering Notes
ACM SIGSOFT Software Engineering Notes  Volume 39, Issue 1
January 2014
193 pages
ISSN:0163-5948
DOI:10.1145/2557833
  • Editor:
  • Will Tracz
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 February 2014
Published in SIGSOFT Volume 39, Issue 1

Check for updates

Author Tags

  1. experimentation
  2. methods
  3. replication
  4. reporting
  5. software engineering
  6. validation
  7. validity

Qualifiers

  • Review-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2015)Six strategies for generalizing software engineering theoriesScience of Computer Programming10.1016/j.scico.2014.11.013101(136-152)Online publication date: Apr-2015
  • (2015)Investigations about replication of empirical studies in software engineeringInformation and Software Technology10.1016/j.infsof.2015.02.00164:C(76-101)Online publication date: 1-Aug-2015

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media