Skip to main content
SpringerLink
Log in

Assessing publication performance of research units: extensions through operational research and economic techniques

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Many quantitative measures exist to assess the publishing outputs of research units such as university departments or institutes. In addition to well-known issues with such measures, further shortcomings include inadequate adjustments for relative entity sizes and researcher intensity, the extent to which research is concentrated among a few rather than all researchers and lags between staffing and publication. This article presents a further array of possible measurement indices, based on operational research and economic ratios, which are capable of adjusting for each of these shortcomings, and which analysts can combine with relatively little effort into existing measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alewell, K. (1990). Criteria for performance profiles of departments and universities. Scientometrics, 19(5–6), 337–347.

    Article  Google Scholar 

  • Allison, P. (1980). Inequality and scientific productivity. Social Studies of Science, 10, 163–179.

    Article  Google Scholar 

  • Baird, L. (1991). Publication productivity in doctoral research departments: Interdisciplinary and intradisciplinary factors. Research in Higher Education, 32, 303–318.

    Article  Google Scholar 

  • Brown, R. G., Meyer, R. F., & D’Esopo, D. A. (1961). The fundamental theorem of exponential smoothing. Operations Research, 9(5), 673–687.

    Article  MATH  MathSciNet  Google Scholar 

  • Carpenter, M. P., Gibb, F., Harris, M., Irvine, J., Martin, B. R., & Narin, F. (1988). Bibliometric profiles for British academic institutions: An experiment to develop research output indicators. Scientometrics, 14(3–4), 213–233.

    Article  Google Scholar 

  • Colman, A. M., Dhillion, D., & Coulthard, B. (1995). A bibliometric evaluation of the research performance of British university politics departments: Publications in leading journals. Scientometrics, 32(1), 49–66.

    Article  Google Scholar 

  • Crewe, I. (1988). Reputation, research, reality: The publication records of UK departments of politics 1978–1984. Scientometrics, 14(3–4), 235–250.

    Article  Google Scholar 

  • Donovan C. (2005). A review of current Australian and International practice in measuring the quality and impact of publicly funded research in the humanities, arts, and social sciences. Discussion paper 05/3, Research School of Social Sciences, Australian National University.

  • Dundar, H., & Lewis, D. R. (1998). Determinants of research productivity in higher education. Research in Higher Education, 39(6), 607–631.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  Google Scholar 

  • Johnes, G., & Johnes, J. (1992). Apples, oranges: The aggregation problem in publications analysis. Scientometrics, 25(2), 353–365.

    Article  Google Scholar 

  • Krampen, G. (2008). The evaluation of university departments and their scientists. Scientometrics, 76(1), 3–21.

    Article  Google Scholar 

  • Lewinson, G. (2001). Evaluation of books as research outputs in history. Research Evaluation, 10(2), 89–95.

    Article  Google Scholar 

  • Lotka, G. (1926). The frequency distribution of scientific productivity. Journal of the Washington Academy of Sciences, 16, 317.

    Google Scholar 

  • Martin, B. R. (1996). The use of multiple indicators in the assessment of basic research. Scientometrics, 36, 343–362.

    Article  Google Scholar 

  • Nederhof, A. J., Meijer, R. F., Moed, H. F., & Van Raan, A. F. J. (1993). Research performance indicators for university departments: A study of an agricultural university. Scientometrics, 27(2), 157–178.

    Article  Google Scholar 

  • Nederhof, A. J., & Noyons, E. C. M. (1992). Assessment of the international standing of university departments’ research: A comparison of bibliometric methods. Scientometrics, 24(3), 393–404.

    Article  Google Scholar 

  • Okrasa, W. (1987). Differences in scientific productivity. Scientometrics, 12(3–4), 221–239.

    Article  Google Scholar 

  • Patrick, W. J., & Stanley, E. C. (1998). Teaching and research quality indicators and the shaping of higher education. Research in Higher Education, 39(1), 19–41.

    Article  Google Scholar 

  • Qurashi, M. M. (1993). Dependence of publication-rate on size of some university groups and departments in UK and Greece in comparison with N.C.I., USA. Scientometrics, 27(1), 19–38.

    Article  Google Scholar 

  • Ramsden, P. (1994). Describing and explaining research productivity. Higher Education, 28, 207–226.

    Article  Google Scholar 

  • Research Evaluation & Policy Project. (2005) Quantitative indicators for research assessment—A literature review. Discussion paper 05/3, Research School of Social Sciences, Australian National University.

  • Rousseau, R. (2000, October 23–25). Concentration and evenness measures as macro-level scientometric indicators. Paper presented at the 2nd International Seminar on Quantitative Evaluation of Research Performance, Shanghai.

  • Scherer, E. M., & Ross, D. (1990). Industrial market structure and economic performance. Boston: Houghton Mifflin.

    Google Scholar 

  • Schloegl, C., Gorraiz, J., Bart, C., & Bargmann, M. (2003). Evaluating two Austrian university departments: Lessons learned. Scientometrics, 56(3), 287–299.

    Article  Google Scholar 

  • Sternberg, R., & Litzenberger, T. (2005). The publication, citation output of German Faculties of Economics, Social Sciences: A comparison of faculties and disciplines based upon SSCI data. Scientometrics, 65(1), 29–53.

    Article  Google Scholar 

  • Stevenson, W. J., & Ozgur, C. (2007). Introduction to management science. Boston: McGraw-Hill Irwin.

    Google Scholar 

  • Tan, D. (1986). The assessment of quality in higher education: A critical review. Research in Higher Education, 24, 223–265.

    Article  Google Scholar 

  • Tijssen, R. J. W. (2007). Africa’s contribution to the worldwide research literature: New analytical perspectives,trends, and performance indicators. Scientometrics, 71(2), 303–327.

    Article  Google Scholar 

  • Toutkoushian, R. K., Porter, S. R., Danielson, C., & Hollis, P. R. (2003). Using publication counts to measure an institution’s research productivity. Research in Higher Education, 44(2), 121–148.

    Article  Google Scholar 

  • Verbeek, A., DeBackere, K., Luwel, M., & Zimmerman, E. (2002). Measuring progress, evolution in science, technology—I: The multiple uses of bibliometric indicators. International Journal of Management Reviews, 4(2), 179–211.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of the Witwatersrand, Johannesburg, South Africa

    Gregory John Lee

  2. New Commerce Building, Private Bag x3, Wits, 2050, South Africa

    Gregory John Lee

Authors
  1. Gregory John Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gregory John Lee.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lee, G.J. Assessing publication performance of research units: extensions through operational research and economic techniques. Scientometrics 84, 717–734 (2010). https://doi.org/10.1007/s11192-010-0210-z

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-010-0210-z

Keywords

Search

Navigation