Skip to main content
SpringerLink
Log in

Methodi Ordinatio: a proposed methodology to select and rank relevant scientific papers encompassing the impact factor, number of citation, and year of publication

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

An increase in the number of scientific publications in the last few years, which is directly proportional to the appearance of new journals, has made the researchers’ job increasingly complex and extensive regarding the selection of bibliographic material to support their research. Not only is it a time consuming task, it also requires suitable criteria, since the researchers need to elect systematically the most relevant literature works. Thus the objective of this paper is to propose the methodology called Methodi Ordinatio, which presents criteria to select scientific articles. This methodology employs an adaptation of the ProKnow-C for selection of publications and the InOrdinatio, which is an index to rank by relevance the works selected. This index crosses the three main factors under evaluation in a paper: impact factor, year of publication and number of citations. When applying the equation, the researchers identify among the works selected the most relevant ones to be in their bibliographic portfolio. As a practical application, it is provided a research sample on the theme technology transfer models comprising papers from 1990 to 2015. The results indicated that the methodology is efficient regarding the objectives proposed, and the most relevant papers on technology transfer models are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Afonso, M. H. F., de Souza, J. V., Ensslin, S. R., & Ensslin, L. (2012). Como construir conhecimento sobre o tema de pesquisa? Aplicação do processo proknow-c na busca de literatura sobre avaliação do desenvolvimento sustentável. Revista de Gestão Social E Ambiental,. doi:10.5773/rgsa.v5i2.424.

    Google Scholar 

  • Antelman, K. (2004). Do open-access articles have a greater research impact? College & Research Libraries, 65(5), 372–382. doi:10.5860/crl.65.5.372.

    Article  Google Scholar 

  • Barham, B. L., Foltz, J. D., & Prager, D. L. (2014). Making time for science. Research Policy, 43(1), 21–31. doi:10.1016/j.respol.2013.08.007.

    Article  Google Scholar 

  • Bhupatiraju, S., et al. (2012). Knowledge flows: analyzing the core literature of innovation, entrepreneurship and science and technology studies. Research Policy, 41, 1205–1218. doi:10.1016/j.respol.2012.03.011.

    Article  Google Scholar 

  • Bornmann, L. (2010). Towards an ideal method of measuring research performance: Some comments to the Opthof and Leydesdorff (2010) paper. Journal of Informetrics, 4(3), 441–443. doi:10.1016/j.joi.2010.04.004.

    Article  Google Scholar 

  • Bruggemann, R., & Carlsen, L. (2012). Multi-criteria decision analyses. Viewing MCDA in terms of both process and aggregation methods: Some thoughts, motivated by the paper of Huang, Keisler and Linkov. Science of the Total Environment, 425, 293–295.

    Article  Google Scholar 

  • Cho, K. T. (2003). Multicriteria decision methods: An attempt to evaluate and unify. Mathematical and Computer Modelling, 37(9–10), 1099–1119. doi:10.1016/S0895-7177(03)00122-5.

    Article  MathSciNet  MATH  Google Scholar 

  • Cinelli, M., Coles, S. R., & Kirwan, K. (2014). Analysis of the potentials of multicriteria decision analysis methods to conduct sustainability assessment. Ecological Indicators, 46, 138–148.

    Article  Google Scholar 

  • De Greve, J. P., & Frijdal, A. (1989). Evaluation of scientific research profile analysis: a mixed method. Higher Education Management, 1, 83–90.

    Google Scholar 

  • Dieks, D., & Chang, H. (1976). Differences in impact of scientific publications: Some indices derived from a citation analysis. Social Studies of Science, 6, 247–267. doi:10.1177/030631277600600204.

    Article  Google Scholar 

  • Dyer, J. S., Fishburn, P. C., Steuer, R. E., Wallenius, J., & Zionts, S. (1992). Multiple criteria decision making, multiattribute utility theory: The next ten years. Management Science, 38(5), 645–654.

    Article  MATH  Google Scholar 

  • Figueira, J., Mousseau, V., & Roy, B. (2005). Electre methods. In Multiple criteria decision analysis: State of the art surveys. International series in operations research and management science (Vol. 78, pp. 133–153). United States: Springer.

  • Greco, S., Matarazzo, B., & Slowinski, R. (2001a). Rough sets theory for multicriteria decision analysis. European Journal of Operational Research, 129, 1–47.

    Article  MathSciNet  MATH  Google Scholar 

  • Greco, S., Matarazzo, B., Slowinski, R. & Stefanowski, J. (2001b). Variable consistency model of dominance-based rough sets approach. In Rough sets and current trends in computing. Lecture notes in computer science, 2005, 170–18.

  • Haeussler, C., Jiang, L., Thursby, J., & Thursby, M. (2014). Specific and general information sharing among competing academic researchers. Research Policy, 43(3), 465–475. doi:10.1016/j.respol.2013.08.017.

    Article  Google Scholar 

  • Higgins, J.P.T. & Green, S. (Eds.). (2011). Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration, Retrieved July 8, 2015, from www.cochrane-handbook.org.

  • Irvine, J., & Martin, B. R. (1983). Assessing basic research: The case of the Isaac Newton Telescope. Social Studies of Science, 13, 49–86. doi:10.1177/030631283013001004.

    Article  Google Scholar 

  • Keeney, R. L., & Raiffa, H. (1993). Decisions with multiple objectives: Preferences and value trade-offs. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Lacerda, R. T. O., Ensslin, L. & Ensslin, S. R. (2012). A bibliometric analysis of strategy and performance measurement. Gestão & Produção, 19(1), 59–78. Retrieved July 7, 2015, from http://www.scielo.br/pdf/gp/v19n1/a05v19n1.

  • Lacerda, R. T. O., Ensslin, L., & Ensslin, S. R. (2015). Research methods and success meaning in project management. In B. Pasian (Ed.), Designs, methods and practices for research of project management. England: Gower Publishing Ltd.

    Google Scholar 

  • Martin, B. R. (1996). The use of multiple indicators in the assessment of basic research. Scientometrics, 36(3), 343–362. doi:10.1007/BF02129599.

    Article  Google Scholar 

  • Nightingale, A. (2009). A guide to systematic literature reviews. Surgery (Oxford), 27(9), 381–384.

    Article  Google Scholar 

  • Pohekar, S. D., & Ramachandran, M. (2004). Application of multi-criteria decision making to sustainable energy planning: A review. Renewable and Sustainable Energy Reviews, 8, 365–381.

    Article  Google Scholar 

  • Roy, B. (1993). Decision science or decision-aid science? European Journal of Operational Research, 66, 184–203.

    Article  Google Scholar 

  • Roy, B. (2005). Paradigms and challenges. In J. Figueira, S. Greco, & M. Ehrgott (Eds.), Multiple criteria decision analysis: State of the art surveys. Berlin: Springer.

    Google Scholar 

  • Saaty, T. L. (1990). How to make a decision? The analytic hierarchy process. European Journal of Operantional Research, 48, 9–26.

    Article  MATH  Google Scholar 

  • Small, H., Boyack, K. W., & Klavans, R. (2014). Identifying emerging topics in science and technology. Research Policy, 43(8), 1450–1467. doi:10.1016/j.respol.2014.02.005.

    Article  Google Scholar 

  • Taylor, P. J. (2002). A partial order scalogram analysis of communication behavior in crisis negotiation with the prediction of outcome. The International Journal of Conflict Management, 13(1), 4–37.

    Article  Google Scholar 

  • Ul Haq, M. (2003). The birth of the Human Development Index. In A. Kumar (Ed.), Readings in human development (pp. 127–137). Oxford: Oxford University Press.

    Google Scholar 

  • UNDP (2015). United Nations Development Program. Human Development Reports. Retrieved July 9, 2015, from http://hdr.undp.org/.

  • van Raan, A. F. J. (2004). Measuring science. Capita selecta of current main issues. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research. The use of publication and patent statistics in studies of S&T systems (pp. 19–50). Kluwer Academic Publishers: Dordrecht.

    Google Scholar 

  • Vaz, C. R., Tasca, J. E., Ensslin, L., Ensslin, S. R., & Selig, P. M. (2013). Avaliação de desempenho na gestão estratégica organizacional: seleção de um referencial teórico de pesquisa e análise bibliométrica. Revista Gestão Industrial,. doi:10.3895/S1808-04482012000400008.

    Google Scholar 

  • Vinkler, P. (1986a). Management system for a scientific research institute based on the assessment of scientific publications. Research Policy, 15(2), 77–87. doi:10.1016/0048-7333(86)90003-X.

    Article  Google Scholar 

  • Vinkler, P. (1986b). Evaluation of some methods for the relative assessment of scientific publications. Scientometrics, 10, 157–177. doi:10.1007/BF02026039.

    Article  Google Scholar 

  • Vinkler, P. (1996). The use of multiple indicators in the assessment of basic research. Scientometrics, 36(3), 343–362. doi:10.1007/BF02129599.

    Article  Google Scholar 

  • Vinkler, P. (2004). Characterization of the impact of sets of scientific papers: The Garfield (Impact) Factor. Journal of the American Society for Information Science and Technology, 55, 431–435. doi:10.1002/asi.10391.

    Article  Google Scholar 

  • Vinkler, P. (2009). πv-index: A new indicator for assessing scientific impact. Journal of Information Science, 35, 602–612. doi:10.1177/0165551509103601.

    Article  Google Scholar 

  • Vinkler, P. (2010). The πv-index: A new indicator to characterize the impact of journals. Scientometrics, 82, 461–475. doi:10.1007/s11192-010-0182-z.

    Article  Google Scholar 

  • Vinkler, P. (2012). The case of scientometricians with the “absolute relative” impact Indicator. Journal of Informetrics, 6, 254–264. doi:10.1016/j.joi.2011.12.004.

    Article  Google Scholar 

  • Zopounidis, C., & Doumpos, M. (2002). Multicriteria classification and sorting methods: A literature review. European Journal of Operational Research, 138, 229–246.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

We thank the Brazilian Government, the Ministry of Education, and UTFPR that supported this research.

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, Federal University of Technology - Paraná (UTFPR) Câmpus Ponta Grossa, Av. Monteiro Lobato, s/n - Km 04, Ponta Grossa, PR, CEP 84016-210, Brazil

    Regina Negri Pagani, João Luiz Kovaleski & Luis Mauricio Resende

  2. Department of Post-graduation in Production Engineering, Federal University of Technology - Paraná (UTFPR) Câmpus Ponta Grossa, Av. Monteiro Lobato, s/n - Km 04, Ponta Grossa, PR, CEP 84016-210, Brazil

    João Luiz Kovaleski & Luis Mauricio Resende

Authors
  1. Regina Negri Pagani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Luiz Kovaleski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luis Mauricio Resende
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Regina Negri Pagani.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pagani, R.N., Kovaleski, J.L. & Resende, L.M. Methodi Ordinatio: a proposed methodology to select and rank relevant scientific papers encompassing the impact factor, number of citation, and year of publication. Scientometrics 105, 2109–2135 (2015). https://doi.org/10.1007/s11192-015-1744-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-015-1744-x

Keywords

Search

Navigation