Abstract
Number of researchers, journals and articles has significantly increased in the last few years and peer review is still the most reliable instrument to sort out innovative, valuable, scientifically sound information from the pool of submitted results. Editors and publishers join their efforts to improve peer review process and to be able to do so properly, they need “field information” from contributors. Editorial board of the Journal of the Serbian Chemical Society performed its own survey to find out what reviewers recognize as priority criteria in manuscript evaluation and whether the professional background (title, number of years in research or experience in reviewing) influences these criteria. Most reviewers declared that they consider peer review as an essential component of the scientific professionalism. Scientific contribution and originality were the most important criteria in the evaluation of papers. Most reviewers preferred to see conclusions completely supported by experimental data, without additional speculations. Although there were no large differences between early stage and experienced researchers, early stage researchers and less experienced reviewers used grade 5 (indicating the highest priority) much more often in their evaluation of priority criteria than experienced researchers and/or reviewers, suggesting possible evolution of tolerance with experience.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11192-016-1869-6/MediaObjects/11192_2016_1869_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11192-016-1869-6/MediaObjects/11192_2016_1869_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11192-016-1869-6/MediaObjects/11192_2016_1869_Fig3_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11192-016-1869-6/MediaObjects/11192_2016_1869_Fig4_HTML.gif)
Similar content being viewed by others
References
Aleksic, J., Alexa, A., Attwood, T. K., Chue Hong, N., Dahlö, M., Davey, R., et al. (2014). An open science peer review oath. F1000Research, 3, 271. doi:10.12688/f1000research.5686.2.
Ashfold, M., & Appleyard, S. (2004). Standards for the quality and originality of articles in PCCP? A notice to authors and referees. Physical Chemistry Chemical Physics, 6(15), E9. doi:10.1039/b409821h.
Azar, O. H. (2005). The review process in economics: Is it too fast? Southern Economic Journal, 72(2), 482–491. doi:10.2307/20062123.
Bailey, C. D., Hermanson, D. R., & Louwers, T. J. (2008). An examination of the peer review process in accounting journals. Journal of Accounting Education, 26(2), 55–72. doi:10.1016/j.jaccedu.2008.04.001.
Benos, D. J., Bashari, E., Chaves, J. M., Gaggar, A., Kapoor, N., LaFrance, M., et al. (2007). The ups and downs of peer review. Advances in Physiology Education, 31(2), 145–152. doi:10.1152/advan.00104.2006.
Birukou, A., Wakeling, J. R., Bartolini, C., Casati, F., Marchese, M., Mirylenka, K., et al. (2011). Alternatives to peer review: Novel approaches for research evaluation. Frontiers in Computational Neuroscience, 5, 56. http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=22174702
Bornmann, L., Nast, I., & Daniel, H.-D. (2008). Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics, 77(3), 415–432. doi:10.1007/s11192-007-1950-2.
Bosch, X., Hernández, C., Pericas, J. M., Doti, P., & Marušić, A. (2012). Misconduct policies in high-impact biomedical journals. PLoS One, 7(12), e51928. doi:10.1371/journal.pone.0051928.
Clark, R. K. F. (2012). Peer review: A view based on recent experience as an author and reviewer. British Dental Journal, 213(4), 153–154. doi:10.1038/sj.bdj.2012.721.
Day, T. E. (2015). The big consequences of small biases: A simulation of peer review. Research Policy, 44(6), 1266–1270. doi:10.1016/j.respol.2015.01.006.
DeMaria, A. N. (2008). Looking back. Journal of the American College of Cardiology, 52(25), 2211–2212. doi:10.1016/j.jacc.2008.12.001.
Ernst, E., & Resch, K. L. (1994). Reviewer bias: A blinded experimental study. Journal of Laboratory and Clinical Medicine, 124(2), 178–182. http://www.ncbi.nlm.nih.gov/pubmed/8051481
Fein, C. (2013). Multidimensional Journal Evaluation of PLOS ONE. Libri, 63(4), 259–271. doi:10.1515/libri-2013-0021.
García Puig, J., Gaspar Alonso-Vega, G., & Ríos Blanco, J. J. (2012). Report of the editors, 2011. Revista Clínica Española, 212(1), 31–39. doi:10.1016/j.rce.2011.11.004.
Ghosh, S. S., Klein, A., Avants, B., & Millman, K. J. (2012). Learning from open source software projects to improve scientific review. Frontiers in Computational Neuroscience, 6, 18. doi:10.3389/fncom.2012.00018.
Godlee, F., Gale, C. R., & Martyn, C. N. (1998). Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: A randomized controlled trial. JAMA: The Journal of the American Medical Association, 280(3), 237–240. doi:10.1001/jama.280.3.237.
Golden, M., & Schultz, D. M. (2012). Quantifying the volunteer effort of scientific peer reviewing. Bulletin of the American Meteorological Society, 93(3), 337–345. doi:10.1175/BAMS-D-11-00129.1.
Guraya, S. Y. (2014). Accuracy of references in scholarly journals: An analysis of 450 references in ten biomedical journals. European Science Editing, 40(4), 88–90. http://www.scopus.com/inward/record.url?eid=2-s2.0-84920670315&partnerID=tZOtx3y1
Hesman Saey, T. (2015). Is redoing scientific research the best way to find truth? Science News 187(2), January 13. https://www.sciencenews.org/article/redoing-scientific-research-best-way-find-truth
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. doi:10.1038/520429a.
Ho, R. C.-M., Mak, K.-K., Tao, R., Lu, Y., Day, J. R., & Pan, F. (2013). Views on the peer review system of biomedical journals: an online survey of academics from high-ranking universities. BMC Medical Research Methodology, 13, 74. doi:10.1186/1471-2288-13-74.
Hoffmann, R. (2008). A wiki for the life sciences where authorship matters. Nature Genetics, 40(9), 1047–1051. doi:10.1038/ng.f.217.
Holden, C. (2005). Stem cell research. Korean cloner admits lying about oocyte donations. Science, 310(5753), 1402–1403. doi:10.1126/science.310.5753.1402.
Houry, D., Green, S., & Callaham, M. (2012). Does mentoring new peer reviewers improve review quality? A randomized trial. BMC Medical Education, 12, 83. doi:10.1186/1472-6920-12-83.
Kachewar, S. G., & Sankaye, S. B. (2013). Reviewer index: A new proposal of rewarding the reviewer. Mens Sana Monographs, 11(1), 274–284. doi:10.4103/0973-1229.109347.
Kearney, M. H., Baggs, J. G., Broome, M. E., Dougherty, M. C., & Freda, M. C. (2008). Experience, time investment, and motivators of nursing journal peer reviewers. Journal of Nursing Scholarship, 40(4), 395–400. doi:10.1111/j.1547-5069.2008.00255.x.
Kelty, C. M., Burrus, S., & Baranuik, R. G. (2008). Peer review anew: Three principles and a case study in publication quality assurance. Proceedings of the IEEE, 96(6), 1000–1011. doi:10.1109/JPROC.2008.921613.
Kronick, D. A. (1990). Peer review in 18th-century scientific journalism. JAMA: The Journal of the American Medical Association, 263(10), 1321–1322. doi:10.1001/jama.1990.03440100021002.
Laakso, M., & Björk, B.-C. (2012). Anatomy of open access publishing: A study of longitudinal development and internal structure. BMC Medicine, 10(1), 124. doi:10.1186/1741-7015-10-124.
Marchionini, G. (2008). Editorial: Reviewer merits and review control in an age of electronic manuscript management system. ACM Transactions on Information Systems, 26(4), 25. doi:10.1145/1402256.1402264.
Moizer, P. (2009). Publishing in accounting journals: A fair game? Accounting, Organizations and Society, 34(2), 285–304. doi:10.1016/j.aos.2008.08.003.
Moos, D. D., & Hawkins, P. (2009). Barriers and strategies to the revision process from an editor’s perspective. Nursing Forum, 44(2), 79–92. doi:10.1111/j.1744-6198.2009.00131.x.
Nedić, O., & Dekanski, A. (2015). A survey on the Journal of the Serbian Chemical Society publishing policies—On the occasion of the 80th volume. Journal of the Serbian Chemical Society, 80(7), 959–969. doi:10.2298/JSC150306036N.
Oosterhaven, J. (2015). Too many journals? Towards a theory of repeated rejections and ultimate acceptance. Scientometrics, 103(1), 261–265. doi:10.1007/s11192-015-1527-4.
Reuters, T. (2015). IP & Science. http://ip-science.thomsonreuters.com/
Siler, K., Lee, K., & Bero, L. (2014). Measuring the effectiveness of scientific gatekeeping. Proceedings of the National Academy of Sciences, 112(2), 360–365. doi:10.1073/pnas.1418218112.
Souder, L. (2011). The ethics of scholarly peer review: A review of the literature. Learned Publishing, 24(1), 55–74. doi:10.1087/20110109.
Squazzoni, F., Bravo, G., & Takács, K. (2013). Does incentive provision increase the quality of peer review? An experimental study. Research Policy, 42(1), 287–294. doi:10.1016/j.respol.2012.04.014.
Stahel, P. F., & Moore, E. E. (2014). Peer review for biomedical publications: We can improve the system. BMC Medicine, 12, 179. doi:10.1186/s12916-014-0179-1.
Steinhauser, G., Adlassnig, W., Risch, J. A., Anderlini, S., Arguriou, P., Armendariz, A. Z., et al. (2012). Peer review versus editorial review and their role in innovative science. Theoretical Medicine and Bioethics, 33(5), 359–376. doi:10.1007/s11017-012-9233-1.
Vintzileos, A. M., Ananth, C. V., Odibo, A. O., Chauhan, S. P., Smulian, J. C., & Oyelese, Y. (2014). The relationship between a reviewer’s recommendation and editorial decision of manuscripts submitted for publication in obstetrics. American Journal of Obstetrics and Gynecology, 211(6), 703.e1–e5.
Yalow, R. S. (1977). Radioimmunoassay: A probe for fine structure of biologic systems. Nobel Lecture. http://www.nobelprize.org/nobel_prizes/medicine/laureates/1977/yalow-lecture.pdf
Acknowledgments
This research work was inspired by and is part of scientific activities in the COST Action TD1306 “New Frontiers of Peer Review (PEERE)”.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nedić, O., Dekanski, A. Priority criteria in peer review of scientific articles. Scientometrics 107, 15–26 (2016). https://doi.org/10.1007/s11192-016-1869-6
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-016-1869-6