Abstract
We compared three different bibliometric evaluation approaches: two citation-based approaches and one based on manual classification of publishing channels into quality levels. Publication data for two universities was used, and we worked with two levels of analysis: article and department. For the article level, we investigated the predictive power of field normalized citation rates and field normalized journal impact with respect to journal level. The results for the article level show that evaluation of journals based on citation impact correlate rather well with manual classification of journals into quality levels. However, the prediction from field normalized citation rates to journal level was only marginally better than random guessing. At the department level, we studied three different indicators in the context of research fund allocation within universities and the extent to which the three indicators produce different distributions of research funds. It turned out that the three distributions of relative indicator values were very similar, which in turn yields that the corresponding distributions of hypothetical research funds would be very similar.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
For discretion reasons, we will neither reveal the names of the universities nor the names of their departments.
Equivalently, since we only consider articles, how well does cf (jcf) predict w?
Available at http://dbh.nsd.uib.no/kanaler/?search=advanced.
True positive rate and false positive rate correspond to the information retrieval measures recall (proportion of retrieved relevant documents in relation to all relevant documents in the database) and fallout (proportion of retrieved non-relevant documents in relation to all non-relevant documents in the database), respectively.
References
Auranen, O., & Nieminen, M. (2010). University research funding and publication performance-An international comparison. Research Policy, 39(6), 822–834.
Braun, T., & Glänzel, W. (1990). United Germany–the new scientific superpower. Scientometrics, 19(5–6), 513–521.
Butler, L. (2004). What happens when funding is linked to publication counts? In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 389–405). Dordrecht: Kluwer.
Debackere, K., & Glänzel, W. (2004). Using a bibliometric approach to support research policy making: The case of the Flemish BOF-key. Scientometrics, 59(2), 253–276.
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874.
Geuna, A. (2001). The changing rationale for European university research funding: Are there negative unintended consequences? Journal of Economic Issues, 35(3), 607–632.
Geuna, A., & Martin, B. R. (2003). University research evaluation and funding: An international comparison. Minerva, 41(4), 277–304.
Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). New York: Wiley.
Lundberg, J. (2007). Lifting the crown-citation z-score. Journal of Informetrics, 1(2), 145–154.
Moed, H. F., De Bruin, R. E., & van Leeuwen, T. N. (1995). New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications. Scientometrics, 33(3), 381–422.
Opthof, T., & Leydesdorff, L. (2010). Caveats for the journal and field normalizations in the CWTS (“Leiden”) evaluations of research performance. Journal of Informetrics, 4(3), 423–430.
Sandström, U., & Sandström, E. (2009). The field factor: towards a metric for academic institutions. Research Evaluation, 18(3), 243–250.
Schneider, J. W. (2009). An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway. European Political Science, 8(3), 364–378.
Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45(1), 1–11.
Sivertsen, G. (2010). A performance indicator based on complete data for the scientific publication output at research institutions. ISSI Newsletter, 6(1), 22–28.
van Raan, A. F. J. (1996). Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics, 36(3), 397–420.
Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62(1), 117–131.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ahlgren, P., Colliander, C. & Persson, O. Field normalized citation rates, field normalized journal impact and Norwegian weights for allocation of university research funds. Scientometrics 92, 767–780 (2012). https://doi.org/10.1007/s11192-012-0632-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-012-0632-x