skip to main content
10.1145/2591062.2591129acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
Article

Collaborative infrastructure for test-driven scientific model validation

Published: 31 May 2014 Publication History

Abstract

One of the pillars of the modern scientific method is model validation: comparing a scientific model's predictions against empirical observations. Today, a scientist demonstrates the validity of a model by making an argument in a paper and submitting it for peer review, a process comparable to code review in software engineering. While human review helps to ensure that contributions meet high-level goals, software engineers typically supplement it with unit testing to get a more complete picture of the status of a project.
We argue that a similar test-driven methodology would be valuable to scientific communities as they seek to validate increasingly complex models against growing repositories of empirical data. Scientific communities differ from software communities in several key ways, however. In this paper, we introduce SciUnit, a framework for test-driven scientific model validation, and outline how, supported by new and existing collaborative infrastructure, it could integrate into the modern scientific process.

References

[1]
K. Beck. Test Driven Development: By Example. Addison Wesley, 2003.
[2]
G. E. Box and N. R. Draper. Empirical model-building and response surfaces. John Wiley & Sons, 1987.
[3]
J. Howison and J. Herbsleb. Scientific software production: incentives and collaboration. In CSCW, pages 513–522. ACM, 2011.
[4]
A. E. Jinha. Article 50 million: an estimate of the number of scholarly articles in existence. Learned Publishing, 23(3):258–263, July 2010.
[5]
R. Jolivet, F. Schürmann, T. Berger, R. Naud, W. Gerstner, and A. Roth. The quantitative single-neuron modeling competition. Biological Cybernetics, 99(4):417–426, Nov. 2008.
[6]
C. F. Kemerer and M. C. Paulk. The impact of design and code reviews on software quality: An empirical study based on PSP data. IEEE Transactions on Software Engineering, 35(4):534–550, 2009.
[7]
F. Perez and B. E. Granger. IPython: a system for interactive scientific computing. Computing in Science & Engineering, 9(3):21––29, 2007.
[8]
M. F. Sanner et al. Python: a programming language for software integration and development. J Mol Graph Model, 17(1):57–61, 1999.
[9]
J. Segal. Models of scientific software development. In SECSE, May 2008.
[10]
H. Siy and L. Votta. Does the modern code inspection have value? In ICSM, pages 281–289, 2001.

Cited By

View all
  • (2024)Synaptic and dendritic architecture of different types of hippocampal somatostatin interneuronsPLOS Biology10.1371/journal.pbio.300253922:3(e3002539)Online publication date: 12-Mar-2024
  • (2023)Editorial: Reproducibility in neuroscienceFrontiers in Integrative Neuroscience10.3389/fnint.2023.127181817Online publication date: 25-Aug-2023
  • (2023)On the Role of Theory and Modeling in NeuroscienceThe Journal of Neuroscience10.1523/JNEUROSCI.1179-22.202243:7(1074-1088)Online publication date: 15-Feb-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software Engineering
May 2014
741 pages
ISBN:9781450327688
DOI:10.1145/2591062
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

  • TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. cyberinfrastructure
  2. model validation
  3. unit testing

Qualifiers

  • Article

Conference

ICSE '14
Sponsor:

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)2
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Synaptic and dendritic architecture of different types of hippocampal somatostatin interneuronsPLOS Biology10.1371/journal.pbio.300253922:3(e3002539)Online publication date: 12-Mar-2024
  • (2023)Editorial: Reproducibility in neuroscienceFrontiers in Integrative Neuroscience10.3389/fnint.2023.127181817Online publication date: 25-Aug-2023
  • (2023)On the Role of Theory and Modeling in NeuroscienceThe Journal of Neuroscience10.1523/JNEUROSCI.1179-22.202243:7(1074-1088)Online publication date: 15-Feb-2023
  • (2023)A Practical Guide to Reproducible Modeling for Biochemical NetworksComputational Modeling of Signaling Networks10.1007/978-1-0716-3008-2_5(107-138)Online publication date: 20-Apr-2023
  • (2022)Combining hypothesis- and data-driven neuroscience modeling in FAIR workflowseLife10.7554/eLife.6901311Online publication date: 6-Jul-2022
  • (2022)Reproducing and quantitatively validating a biologically-constrained point-neuron model of CA1 pyramidal cellsFrontiers in Integrative Neuroscience10.3389/fnint.2022.104142316Online publication date: 8-Nov-2022
  • (2022)Automatic Reuse, Adaption, and Execution of Simulation Experiments via Provenance PatternsACM Transactions on Modeling and Computer Simulation10.1145/356492833:1-2(1-27)Online publication date: 27-Sep-2022
  • (2021)HippoUnit: A software tool for the automated testing and systematic comparison of detailed models of hippocampal neurons based on electrophysiological dataPLOS Computational Biology10.1371/journal.pcbi.100811417:1(e1008114)Online publication date: 29-Jan-2021
  • (2020)Software engineering practices for scientific software development: A systematic mapping studyJournal of Systems and Software10.1016/j.jss.2020.110848(110848)Online publication date: Oct-2020
  • (2019)Resources for Modeling in Computational NeuroscienceHippocampal Microcircuits10.1007/978-3-319-99103-0_24(807-830)Online publication date: 5-Feb-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media