Skip to main content
SpringerLink
Log in

Rainbow ranking: an adaptable, multidimensional ranking method for publication sets

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Various scientometric indices have been proposed in an attempt to express the quantitative and qualitative characteristics of scientific output. However, fully capturing the performance and impact of a scientific entity (author, journal, institution, conference, etc.) still remains an open research issue, as each proposed index focuses only on particular aspects of scientific performance. Therefore, scientific evaluation can be viewed as a multi-dimensional ranking problem, where dimensions represent the assorted scientometric indices. To address this problem, the skyline operator has been proposed in Sidiropoulos et al. (J Informetr 10(3):789–813, 2016) with multiple combinations of dimensions. In the present work, we introduce a new index derived from the utilization of the skyline operator, called Rainbow Ranking or RR-index that assigns a category score to each scientific entity instead of producing a strict ordering of the ranked entities. Our RR-index allows the combination of any known indices depending on the purposes of the evaluation and outputs a single number metric expressing multi-criteria relative ranking and can be applied to any scientific entity such as authors and journals. The proposed methodology was experimentally evaluated using a dataset of over 105,000 scientists from the Computer Science field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. We appreciate the offer of Microsoft to gratis provide their database API. The API that was used in this work has been discontinued by Microsoft in the summer of 2016.

  2. http://dblp.uni-trier.de/.

References

  • Alguliev, R. M., Aliguliyev, R. M., Fataliyev, T. K., & Hasanova, R. S. (2014). Weighted consensus index for assessment of the scientific performance of researchers. Collnet Journal of Scientometrics and Information Management, 8(2), 371–400.

    Article  Google Scholar 

  • Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). hg-index: A new index to characterize the scientific output of researchers based on h- and g-index. Scientometrics, 82, 391–400.

    Article  Google Scholar 

  • Anderson, T. R., Hankin, R. K., & Killworth, P. D. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.

    Article  Google Scholar 

  • Bollen, J., van de Sompel, H., & Chute, R. (2009). A principal component analysis of 39 scientific impact measures. PLoS ONE, 4(6), e6022.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. P. (2014). h-index research in scientometrics: A summary. Journal of Informetrics, 8(3), 749–750.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., Ηug, S. E., & Daniel, H. (2011). A multilevel meta-analysis of studies reporting correlations between the h-index and 37 different h-index variants. Journal of Informetrics, 5(3), 346–359.

    Article  Google Scholar 

  • Börzsönyi, S., Kossmann, D., & Stocker, K. (2001). The skyline operator. In Proceedings of the IEEE international conference on data engineering (ICDE) (pp. 421–430).

  • Brzezinski, M. (2015). Power laws in citation distributions: Evidence from scopus. Scientometrics, 103(1), 213–228.

    Article  Google Scholar 

  • Chomicki, J., Godfrey, P., Gryz, J., & Liang, D. (2003). Skyline with presorting. In Proceedings of the IEEE international conference on data engineering (ICDE) (pp. 717–719).

  • Egghe, L. (2006). Theory and practice of g-index. Scientometrics, 69(1), 131–152.

    Article  Google Scholar 

  • Franceschini, F. (2014). The citer-success-index: A citer-based indicator to select a subset of elite papers. Scientometrics, 101(2), 963–983.

    Article  Google Scholar 

  • Glanzel, W., Thijs, B., & Debackere, K. (2014). The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment. Scientometrics, 101(2), 939–952.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  MATH  Google Scholar 

  • Hutchins, B., & Yuan, X. A. (2016). Relative citation ratio (RCR): A new metric that uses citation rates to measure influence at the article level. PLoS Biology, 14(9), e1002541.

    Article  Google Scholar 

  • Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The r- and ar-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.

    Article  Google Scholar 

  • Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.

    Google Scholar 

  • Ruane, F., & Tol, R. (2008). Rational (successive) h-indices: An application to economics in the Republic of Ireland. Scientometrics, 75(2), 395–405.

    Article  Google Scholar 

  • Schreiber, M., Malesios, C., & Psarakis, S. (2012). Exploratory factor analysis for the hirsch index, 17 h-typevariants, and some traditional bibliometric indicators. Journal of Informetrics, 6(3), 347–358.

    Article  Google Scholar 

  • Sidiropoulos, A., Gogoglou, A., Katsaros, D., & Manolopoulos, Y. (2016). Gazing at the skyline for star scientists. Journal of Informetrics, 10(3), 789–813.

    Article  Google Scholar 

  • Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253–280.

    Article  Google Scholar 

  • Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2015). Ranking and identifying influential scientists versus mass producers by the Perfectionism Index. Scientometrics, 103(1), 1–31.

    Article  Google Scholar 

  • Stoupas, G. (2017). Citation graph analysis and application of prototype evaluation and ranking methods. Master Thesis. Alexander Technological Institute of Thessaloniki.

  • Tamar, V. L., & Tim, S. E. (2015). Ranking journals using altmetrics. CoRR. abs/1507.00451.

  • Tsai, C. F. (2014). Citation impact analysis of top ranked computer science journals and their rankings. Journal of Informetrics, 8(2), 318–328.

    Article  Google Scholar 

  • Van Raan, A. F. J. (2004). Sleeping beauties in science. Scientometrics, 59(3), 467–472.

    Article  Google Scholar 

  • Voorneveld, M. (2003). Characterization of Pareto dominance. Operations Research Letters, 31(1), 7–11.

    Article  MathSciNet  MATH  Google Scholar 

  • Wildgaard, L., Schneider, J. W., & Larsen, B. (2014). A review of the characteristics of 108 author-level bibliometric indicators. Scientometrics, 101(1), 125–158.

    Article  Google Scholar 

  • Wohlin, C. (2009). A new index for the citation curve of researchers. Scientometrics, 81(2), 521–533.

    Article  Google Scholar 

  • Wolcott, H. N., Fouch, M. J., Hsu, E., Bernaciak, C., Corrigan, J., & Williams, D. (2015). Modeling time-dependent and -indpendent indicators to facilitate identification of breakthrough research papers. In Proceedings of the international society of scientometrics & informetrics conference (ISSI) (pp. 403–408).

  • Zhang, C. (2009). The e-index, complementingthe h-index for excess citations. PLoS ONE, 4(5), e5429.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Informatics, Aristotle University, 54124, Thessaloniki, Greece

    Georgios Stoupas, Antonia Gogoglou & Yannis Manolopoulos

  2. Department of Information Technology, Alexander Technological Educational Institute of Thessaloniki, Thessaloniki, Greece

    Antonis Sidiropoulos

  3. Department of Electrical and Computer Engineering, University of Thessaly, Volos, Greece

    Dimitrios Katsaros

Authors
  1. Georgios Stoupas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonis Sidiropoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonia Gogoglou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dimitrios Katsaros
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yannis Manolopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Georgios Stoupas.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stoupas, G., Sidiropoulos, A., Gogoglou, A. et al. Rainbow ranking: an adaptable, multidimensional ranking method for publication sets. Scientometrics 116, 147–160 (2018). https://doi.org/10.1007/s11192-018-2731-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-018-2731-9

Keywords

Search

Navigation