Abstract
As concern over the use of questionable research practices (QRPs) in academic science has increased over the last couple of decades, some reforms have been implemented and many others have been debated and recommended. While many of these proposals have merit, efforts to improve scientific practices are more likely to succeed when they are responsive to the prevailing views and concerns of scientists themselves. To date, there have been few efforts to solicit wide-ranging input from researchers on the topic of needed reforms. This article is a qualitative report of responses from federally funded scientists to the question of what should be done to address the problem of QRPs in their disciplines. Overall, participants were concerned about how institutional and career-oriented incentives encourage the use of QRPs. Compared to previous recommendations, participants had surprisingly little confidence in the ability of ethics training to improve research integrity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Results from Likert-type responses regarding 31 QRPs reported in Bruton et al. forthcoming.
Minor edits to correct misspellings and other incidental mistakes in narrative responses have been made to improve readability and comprehension.
References
6th World Conference on Research Integrity. (6th WCRI). (2019). The Hong Kong manifesto for assessing researchers: Fostering research integrity. (2 Sep 2019 version). Retrieved September 7, 2019, from http://wcri2019.org/uploads/files/2019_new/Hong_Kong_Manifesto_0902.pdf.
Alberts, B., Cicerone, R. J., Fienberg, S. E., Kamb, A., McNutt, M., Nerem, R. M., et al. (2015). Scientific integrity. Self-correction in science at work. Science,348(6242), 1420–1422.
American Society for Cell Biology (2013). DORA. Declaration on research assessment. Retrieved August 28, 2019, from https://sfdora.org/read/.
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., et al. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality,27(2), 108–119.
Baccini, A., De Nicolau, G. D., & Petrovish, E. (2019). Citation gaming induced by bibiliometric evaluation: A country-level comparative analysis. PLoS One,14(9), e0221212. https://doi.org/10.1371/journal.pone.0221212.
Begley, C. G., & Ioannidis, J. P. A. (2015). Reproducibility in science: Improving the standard for basic and preclinical research. Circulation Research, 116, 116–126.
Blankstein, M., & Wolff-Eisenberg, C. (2019). Ithaka S + R US Faculty Survey 2018. 12 April. Retrieved August 27, 2019, from https://doi.org/10.18665/sr.311199.
Bruton, S., Brown, M., & Sacco, D. F. (forthcoming). Ethical consistency and experience: An attempt to influence researcher attitudes toward questionable research practices through reading prompts. Journal of Empirical Research on Human Research Ethics. https://doi.org/10.1177/1556264619894435.
Casadevall, A., & Fang, F. C. (2012). Reforming science: Methodological and cultural reforms. Infection and Immunity,80(3), 891–896.
Casadevall, A., & Fang, F. C. (2018). Making the scientific literature fail-safe. The Journal of Clinical Investigation,128(10), 4243–4244.
Chambers, C. (2014). Registered reports: A step change in scientific publishing. 13 November 2014. Retrieved August 21, 2019 from www.elsevier.com/reviewers-update/story/innovation-in-publishing/registered-reports-a-step-change-in-scientific-publishing.
Chambers, C. (2019). The registered reports revolution: Lessons in cultural reform. Significance,16, 23–27.
De Vries, R., Anderson, S., & Martinson, B. C. (2006). Normal misbehavior: Scientists talk about the ethics of research. Journal of Empiricial Research on Human Research Ethics,1(1), 43–50.
Eklund, A., Nichols, T. E., & Knutsson, H. (2016). Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Proceedings of the National Academy of Sciences USA,113, 7900–7905.
Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing,62(1), 107–115.
Fanelli, D. (2012). Negative results are disappearing from most disciplines and countries. Scientometrics,90, 891–904.
Fang, F. C., & Casadevall, A. (2012). Reforming science: Structural reforms. Infection and Immunity,80(3), 897–901.
Gilmore, R. O., Diaz, M. T., Wyble, B. A., & Yarkoni, T. (2017). Progress toward openness, transparency, and reproducibility in cognitive neuroscience. Annals of the New York Academy of Sciences,1396(1), 5–18.
Hales, A. H., Wesselman, E. D., & Hilgard, J. (2019). Improving psychological science through transparency and openness: An overview. Perspectives on Behavior Science,42(1), 13–31.
Hantula, D. A. (2019). Editorial: Replication and reliability in behavior science and behavior analysis: A call for a conversation. Perspectives on Behavior Science,42(1), 1–11.
Hicks, D., Wouters, P., Waltman, L., de Rijke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature,520(7548), 429–431.
Higginson, A. D., & Munafò, M. R. (2016). Current incentives for scientists lead to underpowered studies with erroneous conclusions. PLoS Biology,14(11), e2000995. https://doi.org/10.1371/journal.pbio.2000995.
Holm, S., & Hofmann, B. (2018). Associations between attitudes towards scientific misconduct and self-reported behavior. Accountability in Research,25(5), 290–300.
Hubbard, R. (2016). Corrupt research: The case for reconceptualizing empirical management and social science. Thousand Oaks: Sage Publications.
Ioannidis, J. P. A., Fanelli, D., Dunne, D. D., et al. (2015). Meta-research: Evaluation and improvement of research methods and practices. PLoS Biology,13(10), e1002264. https://doi.org/10.1371/journal.pbio.1002264.
Johnsson, L., Eriksson, S., Helgesson, G., et al. (2014). Making researchers moral: Why trustworthiness requires more than ethics guidelines and review. Research Ethics,10(1), 29–46.
Kidwell, M. C., Lazarević, L. B., Baranski, E., Harwicke, T. E., Piechowski, S., Falkenberg, L., et al. (2016). Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency. PLoS Biology,14, e1002456. https://doi.org/10.1371/journal.pbio.1002456.
Killeen, P. R. (2019). Predict, control, and replicate to understand: How statistics can foster the fundamental goals of science. Perspectives on Behavior Science,42(1), 109–132.
Kretser, A., Murphy, D., Bertuzzi, S., Abraham, T., Allison, D. B., Boor, K. J., et al. (2019). Scientific integrity principles and best practices: Recommendations from a scientific integrity consortium. Science and Engineering Ethics,25(2), 327–355.
Lilienfeld, S. (2017). Psychology’s replication crisis and the grant culture: Righting the ship. Perspectives on Psychological Science,12(4), 660–664.
Malički, M., Aalbersberg, I. J., Bouter, L., & ter Riet, G. (2019). Journals’ instructions to authors: A cross-sectional study across scientific disciplines. PLoS One,14(9), e0222157. https://doi.org/10.1371/journal.pone.0222157.
Matthews, D. (2019). Do researchers trust each other’s work? Survey of more than 3,000 academics finds many are skeptical about scholarship they come across. Times Higher Education, August 27. Retrieved September 7, 2019, from https://www.timeshighereducation.com/news/do-researchers-trust-each-others-work.
Moher, D., Naudet, F., Cristea, I. A., Medema, F., Ioannidis, J. P. A., & Goodman, S. N. (2018). Assessing scientists for hiring, promotion, and tenure. PLoS Biology,16(3), e2004089. https://doi.org/10.1371/journal.pbio.2004089.
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., et al. (2017). A manifesto for reproducible science. Nature Human Behavior,1, 1–9. https://doi.org/10.1038/s41562-016-0021.
National Academies of Sciences, Engineering, and Medicine. (2017). Fostering integrity in research. Washington, DC: The National Academies Press.
National Academies of Sciences, Engineering, and Medicine. (2019). Reproducibility and replicability in science. Washington, DC: The National Academies Press.
Nosek, B. A., Alter, G., Banks, G. C., Boorsboom, D., Bowman, S. D., Breckler, S. J., et al. (2015). Promoting an open research culture. Science,348, 1422–1425.
Nosek, B. A., & Bar-Anan, Y. (2012). Scientific utopia: I. Opening scientific communication. Psychological Inquiry,23(3), 217–243.
Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science,7, 615–631.
Pew Research Center. (2019). Trust and mistrust in Americans’ views of scientific experts. 2 August. Retrieved September 9, 2019, from https://www.pewresearch.org/science/2019/08/02/trust-and-mistrust-in-americans-views-of-scientific-experts/.
Poldrack, R. A. (2019). The costs of reproducibility. Neuron,101(1), 11–14.
Sacco, D. F., Bruton, S. V., & Brown, M. (2018). In defense of the questionable: Defining the basis of research scientists’ engagement in questionable research practices. Journal of Empirical Research on Human Research Ethics,13(1), 1–10.
Samota, E. K., & Davey, R. P. (2019). Knowledge and attitudes among life scientists towards reproducibility within journal articles. BioRxvi, Preprint posted 20 March 2019. https://doi.org/10.1101/581033.
Sijtsma, K. (2016). Playing with data—Or how to discourage questionable research practices and stimulate researchers to do things right. Psychometrika,81(1), 1–15.
Smaldino, P. E., & McElreath, R. (2016). The natural selection of bad science. Royal Society of Open Science,3, 160384. https://doi.org/10.1098/rsos.160384.
Sumpter, J. (2019). Licence to publish will restore trust in science. Times Higher Eduction, 9 August. Retrieved September 9, 2019, from https://www.timeshighereducation.com/opinion/licence-publish-will-restore-trust-science.
Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation,27(2), 237–246.
Tsipursky, G. (2018). (Dis)trust in Science. Psychology Today (blog post, July 5). Retrieved December 17, 2019, from https://www.psychologytoday.com/us/blog/intentional-insights/201807/distrust-in-science.
Ware, J. J., & Munafò, M. R. (2015). Significance chasing in research practice: Causes, consequences, and possible solutions. Addiction,110, 4–8.
Washburn, A. N., Hanson, B. E., Motyl, M., Skitka, L. J., Tantis, C., Wong, K. M., et al. (2018). Why do some psychology researchers resist adopting proposed reforms to research practices? A description of researchers’ rationales. Advances in Methods and Practices in Psychological Science,1(2), 166–173. https://doi.org/10.1177/2515245918757427.
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology,7, 1832. https://doi.org/10.3389/fpsyg.2016.01832.
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data,3(1), 160018. https://doi.org/10.1038/sdata.2016.18.
Acknowledgements
The authors thank our participants for their thoughtful comments and assistance. Funding support received from U.S. Department of Health and Human Services’ Office of Research Integrity (Grant No. 1 ORIIR170035-01-00).
Funding
This work was supported by: U.S. Department of Health and Human Services’ Office of Research Integrity (Grant No. 1 ORIIR170035-01-00).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflicts of interest.
Ethics Approval
All procedures performed were in accordance with the relevant institutional ethical standards (USM IRB #18102605) and with the 1964 Helsinki declaration and its later amendments.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Bruton, S.V., Medlin, M., Brown, M. et al. Personal Motivations and Systemic Incentives: Scientists on Questionable Research Practices. Sci Eng Ethics 26, 1531–1547 (2020). https://doi.org/10.1007/s11948-020-00182-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11948-020-00182-9