Skip to main content
SpringerLink
Log in

Bibliometrics as a tool for measuring gender-specific research performance: an example from South African invasion ecology

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Citations to published work are gaining increasing prominence in evaluations of the research performance of scientists. Considering the importance accorded to gender issues in South African science, it is surprising that (to our knowledge) no research has as yet ascertained the extent of sex differences in citations to the published work of scientists in this country. Our literature study shows that studies that have been conducted elsewhere tend to neglect in their analyses important gender-related and other factors, such as the sex composition of multi-authored papers and the extent of foreign co-authorship. Against this background, we illustrate the difficulties inherent in measuring the quality aspect of sex-specific research performance by means of an analysis of a dataset of articles (n = 229) that were published between 1990 and 2002 in the field of invasion ecology and in journals included in the Thomson Reuters Web of Science. Each article has at least one South African author address. The results indicate that foreign co-authorship is a better correlate of high citations than the sex of South African authors, and this is true irrespective of whether the annual citation rate or window period is used, whether or not self-citations are excluded, and whether or not the number of authors is controlled for by calculating fractional counts. The paper highlights these and other considerations that are relevant for future gender-focused bibliometric research, both in South Africa and beyond.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Mauleón and Bordons (2006), Wennerås and Wold (1997), Toutkoushian (1994) and Trimble (1985, 1993).

  2. Lewison (2001), Davis and Astin (1987), Irvine and Martin (1986), Cole (1979), Cole and Cole (1973), Cole and Zuckerman (1984) and Reskin (1978).

  3. Walters (2006), Haslam et al. (2008) and Montpetit et al. (2008).

  4. Also referred to as “normal counts” (Trifunac 2006, p. 1070).

  5. Also referred to as “adjusted counts” (Cronin and Overfelt 1994, p. 61), or “per author counts” (Trifunac 2006, p. 1070).

  6. In the latter case, however, the sex differences were small enough to lead Over and Moore (1980, p. 415) to conclude that “[t]he men and women were […] equally likely to employ authorship patterns that enhance individual visibility”.

  7. Already in the early 1980s, ISI recognised the need “to treat all authors in every article as though they were listed first” (Garfield 1981, p. 269) and started developing a system that would allow this. In 1978, Garfield (1981) was the first to use these so-called “all-author data”. He does mention that “[d]eveloping all-author lists is considerably more complex than conducting first-author studies” (p. 269). However, his findings (the average author received 1,178 citations to papers on which he or she appeared as first author and 2,633 as co-author) clearly demonstrate “the importance of all-author data” (p. 275).

  8. In some countries, it is possible to differentiate between the sexes based on the surname alone. For instance, in Poland more than sixty percent of all surnames can be determined as belonging to a man (those which end with “-ski”, “-cki” or “-owy”) or a woman (those ending with “-ska”, “-cka” or “-owa”) (Webster 2001). In Iceland, women’s surnames typically end in “dottir”, whereas the names of men end in “son”, which allowed Lewison (2001) to conduct one of the most extensive and detailed analyses of gender differences in citations thus far. Lewison (2001, p. 42) rightly comments that, “It would be very difficult to extend such a study to other countries unless there were a complete list of the country’s researchers available with their sexes so that comparisons could be made”.

  9. See Boshoff (2005) for South African statistics in this regard.

  10. Prof David Richardson, currently Deputy Director of the CIB and a Professor in the Department of Botany and Zoology at Stellenbosch University.

  11. The South African Department of Science and Technology (DST) and National Research Foundation (NRF)—the two bodies responsible for the funding and administration of the Centres of Excellence.

  12. Examples include invasive/invasion (management); alien/plant invasions; naturalized/non-indigenous; indigenous; native; exotic; biological invasions, alien species, invasive alien species and invasion biology/ecology. In subsequent searches, the net was thrown wider, as broader terms and their derivatives (e.g., biological diversity and biological control) were used to identify papers that may have been overlooked by the more obvious key terms. In case of doubt, the contents of papers were scanned in order to decide whether they should be included or not.

References

  • Aksnes, D. W., Rorstad, K., Piro, F., & Sivertsen, G. (2011). Are female researchers less cited? A large-scale study of Norwegian scientists. Journal of the American Society for Information Science and Technology, 62(4), 628–636.

    Article  Google Scholar 

  • Beaver, D. B. (1986). Collaboration and teamwork in physics. Czechoslovak Journal of Physics, 1, 14–18.

    Article  Google Scholar 

  • Bordons, M., Morillo, F., Fernández, M. T., & Gómez, I. (2003). One step further in the production of bibliometric indicators at the micro level: Differences by gender and professional category of scientists. Scientometrics, 57(2), 159–173.

    Article  Google Scholar 

  • Bornmann, L. (2010). Towards an ideal method of measuring research performance: Some comments to the Opthof and Leydesdorff (2010) paper. Journal of Informetrics, 4, 441–443.

    Article  Google Scholar 

  • Borrego, Á., Barrios, M., Villarroya, A., & Ollé, C. (2010). Scientific output and impact of postdoctoral scientists: A gender perspective. Scientometrics, 83, 93–101.

    Article  Google Scholar 

  • Borsuk, R. M., Budden, A. E., Leimu, R., Aarssen, L. W., & Lortie, C. J. (2009). The influence of author gender, national language and number of authors on citation rate in ecology. The Open Ecology Journal, 2, 25–28.

    Google Scholar 

  • Boshoff, N. (2005). The representation of women academics in higher education in South Africa: Progress in the pipeline? South African Journal of Higher Education, 19(2), 359–377.

    Google Scholar 

  • Cole, J. R. (1979). Fair science: Women in the scientific community. New York: Free Press.

    Google Scholar 

  • Cole, J. R., & Cole, S. (1973). Social stratification in science. Chicago: University of Chicago Press.

    Google Scholar 

  • Cole, J. R., & Zuckerman, H. (1984). The productivity puzzle: Persistence and change in patterns of publication of men and women scientists. Advances in Motivation and Achievement, 2, 217–258.

    Google Scholar 

  • Corley, E. A., & Sabharwal, M. (2010). Scholarly collaboration and productivity patterns in public administration: Analysing recent trends. Public Administration, 88(3), 627–648.

    Article  Google Scholar 

  • Craig, I. D., Plume, A. M., McVeigh, M. E., Pringle, J., & Amin, M. (2007). Do open access articles have greater citation impact? A critical review of the literature. Journal of Informetrics, 1(3), 239–248.

    Article  Google Scholar 

  • Creamer, E. G. (1998). Assessing faculty publication productivity: Issues of equity. ASHE-ERIC Higher Education Report (Vol. 26, No. 2). Washington, DC: Association for the Study of Higher Education.

  • Cronin, B., & Overfelt, K. (1994). Citation-based auditing of academic performance. Journal of the American Society for Information Science, 45(2), 61–72.

    Article  Google Scholar 

  • Davenport, E., & Snyder, H. (1995). Who cites women? Whom do women cite? An exploration of gender and scholarly citation in sociology. Journal of Documentation, 51(4), 404–410.

    Article  Google Scholar 

  • Davis, D. E., & Astin, H. S. (1987). Reputational standing in academe. The Journal of Higher Education, 58, 261–275.

    Article  Google Scholar 

  • Demetrulias, D. M. (1986). Gender differences in co-authorship. Journal of Educational Equity and Leadership, 6(2), 119–127.

    Google Scholar 

  • Drake, J., Mooney, H. A., Di Castri, F., Groves, R., Kruger, F. J., Rejmánek, M., et al. (Eds.). (1989). Biological invasions: A global perspective. Chichester: Wiley.

    Google Scholar 

  • Ferber, M. A. (1986). Citations: Are they an objective measure of scholarly merit? Signs: Journal of Women in Culture and Society, 11(2), 381–389.

    Article  Google Scholar 

  • Flores, L. Y., Rooney, S. C., Heppner, P. P., Browne, L. D., & Wei, M.-F. (1999). Trend analyses of major contributions in The Counseling Psychologist cited from 1986 to 1996: Impact and implications. The Counseling Psychologist, 27(1), 73–95.

    Article  Google Scholar 

  • Fox, M. F. (1991). Gender, environmental milieu, and productivity in science. In H. Zuckerman, J. R. Cole, & J. T. Bruer (Eds.), The outer circle: Women in the scientific community (pp. 188–204). New York: Norton.

    Google Scholar 

  • García-Pérez, M. A. (2009). The Hirsch h index in a non-mainstream area: Methodology of the behavioral sciences in Spain. The Spanish Journal of Psychology, 12(2), 833–849.

    Google Scholar 

  • Garfield, E. (1981). The 1,000 contemporary scientists most-cited 1965–1978 (Part I: The basic list and introduction). Current Contents, 41, 5–14. [Reprinted in: Garfield, E. (1981–1982). Essays of an information scientist, 5, 269–278].

  • Gonzalez-Brambila, C., & Veloso, F. M. (2007). The determinants of research output and impact: A study of Mexican researchers. Research Policy, 36, 1035–1051.

    Article  Google Scholar 

  • Hammel, E. A., Mason, C., Prater, A., & Lundy, R. (1995). Gender and the academic career in North American Anthropology: Differentiating intramarket from extramarket bias. Current Anthropology, 36(2), 366–380.

    Article  Google Scholar 

  • Haslam, N., Ban, L., Kaufmann, L., Loughnan, S., Peters, K., Whelan, J., et al. (2008). What makes an article influential? Predicting impact in social and personality psychology. Scientometrics, 76(1), 169–185.

    Article  Google Scholar 

  • Helmreich, R. L., Spence, J. T., Beane, W. E., Lucker, G. W., & Matthews, K. A. (1980). Making it in academic psychology: Personality correlates of attainment. Journal of Personality and Social Psychology, 39(5), 896–908.

    Article  Google Scholar 

  • Housri, N., Cheung, M. C., Koniaris, L. G., & Zimmers, T. A. (2008). Scientific impact of women in academic surgery. Journal of Surgical Research, 148, 13–16.

    Article  Google Scholar 

  • Hunter, L. A., & Leahey, E. (2010). Parenting and research productivity: New evidence and methods. Social Studies of Science, 40(3), 433–451.

    Article  Google Scholar 

  • Hutson, S. R. (2002). Gendered citation practices in American Antiquity and other archaeology journals. American Antiquity, 67(2), 331–342.

    Article  Google Scholar 

  • Hutson, S. R. (2006). Self-citation in archaeology: Age, gender, prestige, and the self. Journal of Archaeological Method and Theory, 13(1), 1–19.

    Article  Google Scholar 

  • Irvine, J., & Martin, B. R. (1986). Women in radio astronomy—Shooting stars? In J. Harding (Ed.), Perspectives on gender and science (pp. 80–103). London: The Falmer Press.

    Google Scholar 

  • Kelly, C. D., & Jennions, M. D. (2006). The h index and career assessment by numbers. Trends in Ecology & Evolution, 21(4), 167–170.

    Article  Google Scholar 

  • Kolpin, V. W., & Singell, L. D., Jr. (1996). The gender composition and scholarly performance of economics departments: A test for employment discrimination. Industrial and Labor Relations Review, 49(3), 408–423.

    Article  Google Scholar 

  • Krampen, G. (2008). The evaluation of university departments and their scientists: Some general considerations with reference to exemplary bibliometric publication and citation analyses for a Department of Psychology. Scientometrics, 76(1), 3–21.

    Article  Google Scholar 

  • Leahey, E. (2007). Not by productivity alone: How visibility and specialization contribute to academic earnings. American Sociological Review, 72(4), 533–561.

    Article  Google Scholar 

  • Leimu, R., & Koricheva, J. (2005). What determines the citation frequency of ecological papers? Trends in Ecology & Evolution, 20(1), 28–32.

    Article  Google Scholar 

  • Leta, J., & Lewison, G. (2003). The contribution of women in Brazilian science: A case study in astronomy, immunology and oceanography. Scientometrics, 57(3), 339–353.

    Article  Google Scholar 

  • Lewison, G. (2001). The quantity and quality of female researchers: A bibliometric study of Iceland. Scientometrics, 52(1), 29–43.

    Article  Google Scholar 

  • Lindsey, D. (1980). Production and citation measures in the sociology of science: The problem of multiple authorship. Social Studies of Science, 10(2), 145–162.

    Article  Google Scholar 

  • Long, J. S. (1992). Measures of sex differences in scientific productivity. Social Forces, 71(1), 159–178.

    Google Scholar 

  • Lutz, C. (1990). The erasure of women’s writing in sociocultural anthropology. American Ethnologist, 17, 611–627.

    Article  Google Scholar 

  • MacRoberts, M. H., & MacRoberts, B. R. (1987). Problems of citation analysis: a critical review. Journal of the American Society for Information Science, 40, 342–349.

    Article  Google Scholar 

  • Mauleón, E., & Bordons, M. (2006). Productivity, impact and publication habits by gender in the area of materials science. Scientometrics, 66(1), 199–218.

    Article  Google Scholar 

  • Mauleón, E., Bordons, M., & Oppenheim, C. (2008). The effect of gender on research staff success in life sciences in the Spanish National Research Council. Research Evaluation, 17(3), 213–225.

    Article  Google Scholar 

  • Mcelhinny, B., Hols, M., Holtzkener, J., Unger, S., & Hicks, C. (2003). Gender, publication and citation in sociolinguistics and linguistic anthropology: The construction of a scholarly canon. Language in Society, 32, 299–328.

    Article  Google Scholar 

  • Montpetit, E., Blais, A., & Foucault, M. (2008). What does it take for a Canadian political scientist to be cited? Social Science Quarterly, 89(3), 802–816.

    Article  Google Scholar 

  • National Advisory Council on Innovation (NACI). (2008). Creating the future South African national system of innovation: Gender, race and SET sector issues. Report by the Science, Engineering and Technology for Women (SET4W), NACI Permanent Sub-Committee. http://www.nacinnovation.biz/about-naci/naci-sub-committees/set4women/set4w-downloads. Accessed 9 March 2011.

  • Over, R., & Moore, D. (1980). Research productivity and impact of men and women in psychology departments of Australian universities, 1975–1977. Australian Psychologist, 15(3), 413–418.

    Article  Google Scholar 

  • Peñas, C. S., & Willet, P. (2006). Gender differences in publication and citation counts in librarianship and information science research. Journal of Information Science, 32(5), 480–485.

    Article  Google Scholar 

  • Persell, C. H. (1983). Gender, rewards and research in education. Psychology of Women Quarterly, 8(1), 33–47.

    Article  Google Scholar 

  • Powell, A., Hassan, T. M., Dainty, A. R. J., & Carter, C. (2009). Exploring gender differences in construction research: A European perspective. Construction Management and Economics, 27, 803–807.

    Article  Google Scholar 

  • Prozesky, H. E. (2006). Gender differences in the journal publication productivity of South African academic authors. South African Review of Sociology, 37(2), 87–112.

    Google Scholar 

  • Pyšek, P., Richardson, D. M., & Jarošik, V. (2006). Who cites who in the invasion zoo: Insights from an analysis of the most highly cited papers in invasion ecology. Preslia, 78, 437–468.

    Google Scholar 

  • Pyšek, P., Richardson, D. M., Pergl, J., Jarošík, V., Sixtová, Z., & Weber, E. (2008). Geographical and taxonomical biases in invasion ecology. Trends in Ecology & Evolution, 23, 237–244.

    Article  Google Scholar 

  • Qiu, H., & Chen, Y.-F. (2009). Bibliometric analysis of biological invasions research during the period of 1991 to 2007. Scientometrics, 81(3), 601–610.

    Article  Google Scholar 

  • Reese-Evans, L. (2010). Gender and citation in two LIS e-journals: A bibliometric analysis of LIBRES and Information Research. Library and Information Science Research Electronic Journal, 20(1), 1–19.

    Google Scholar 

  • Reskin, B. F. (1978). Scientific productivity, sex and location in the institution of science. American Journal of Sociology, 83(5), 1235–1243.

    Article  Google Scholar 

  • Richardson, D. M., & Van Wilgen, B. W. (2004). Invasive alien plants in South Africa: How well do we understand the ecological impacts? South African Journal of Science, 100(Jan/Feb), 45–52.

    Google Scholar 

  • Rousseau, R. (1992). Letter to the editor: Why I am not cited or why are multi-authored papers more cited than others? Journal of Documentation, 48, 79–80.

    Article  Google Scholar 

  • Sandström, U. (2009). Combining curriculum vitae and bibliometric analysis: Mobility, gender and research performance. Research Evaluation, 18(2), 135–142.

    Article  Google Scholar 

  • Sonnert, G. (1995). What makes a good scientist? Determinants of peer evaluation among biologists. Social Studies of Science, 25(1), 35–55.

    Article  Google Scholar 

  • Sonnert, G., & Holton, G. J. (1995). Gender differences in science careers: The Project Access Study. New Brunswick, NJ: Rutgers University Press.

    Google Scholar 

  • Sooryamoorthy, R. (2009). Collaboration and publication: How collaborative are scientists in South Africa? Scientometrics, 80(2), 419–439.

    Article  Google Scholar 

  • Stack, S. (1994). An analysis of the impacts of books and journal articles. International Review of Modern Sociology, 24(2), 119–125.

    Google Scholar 

  • Symonds, M. R. E., Gemmell, N. J., Braisher, T. L., Gorringe, K. L., & Elgar, M. A. (2006). Gender differences in publication output: Towards an unbiased metric of research performance. PLoS ONE, 1(2), e127.

    Article  Google Scholar 

  • Teghtsoonian, M. (1974). Distribution of sex by authors and editors of psychological journals, 1970–1972: Are there enough women editors? American Psychologist, 29, 262–269.

    Article  Google Scholar 

  • Toutkoushian, R. K. (1994). Using citations to measure sex discrimination in faculty scales. The Review of Higher Education, 18(1), 61–82.

    Google Scholar 

  • Tregenza, T. (2002). Gender bias in the refereeing process? Trends in Ecology & Evolution, 17(8), 349–350.

    Article  Google Scholar 

  • Trifunac, M. D. (2006). A note on publication and citation rates of female academics in earthquake engineering. Soil Dynamics and Earthquake Engineering, 26, 1063–1075.

    Article  Google Scholar 

  • Trimble, V. (1985). Some notes on patterns in citations of papers by American astronomers. Quarterly Journal of the Royal Astronomical Society, 26, 40–50.

    Google Scholar 

  • Trimble, V. (1993). Patterns in citations of papers by American astronomers. Quarterly Journal of the Royal Astronomical Society, 34, 235–250.

    Google Scholar 

  • United Nations Educational, Scientific and Cultural Organization (UNESCO). (2007). Science, technology and gender: An international report (executive summary). Science and technology for development series. Paris: UNESCO Publishing. http://unesdoc.unesco.org/images/0015/001540/154027e.pdf. Accessed 9 March 2011.

  • Van Raan, A. F. J. (1998). The influence of international collaboration on the impact of research results. Scientometrics, 42(3), 423–428.

    Article  Google Scholar 

  • Walters, G. D. (2006). Predicting subsequent citations to articles published in twelve crime-psychology journals: Author impact versus journal impact. Scientometrics, 69(3), 499–510.

    Article  MathSciNet  Google Scholar 

  • Walters, C. G., Fry, E. H., & Chaisson, B. D. (1990). Women scholars: Closing the publication gap. Research in Higher Education, 31(4), 355–367.

    Article  Google Scholar 

  • Ward, K. B., Gast, J., & Grant, L. (1992). Visibility and dissemination of women’s and men’s sociological scholarship. Social Problems, 39(3), 291–298.

    Article  Google Scholar 

  • Ward, K. B., & Grant, L. (1991). Gender and publishing in sociology. Gender & Society, 5(2), 207–223.

    Article  Google Scholar 

  • Ward, K. B., & Grant, L. (1996). Gender and academic publishing. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. xi, pp. 172–212). New York: Agathon Press.

    Google Scholar 

  • Webster, B. M. (2001). Polish women in science: A bibliometric analysis of Polish science and its publications, 1980–1999. Research Evaluation, 10(3), 185–194.

    Article  MathSciNet  Google Scholar 

  • Wennerås, C., & Wold, A. (1997). Nepotism and sexism in peer-review. Nature, 387, 341–343.

    Article  Google Scholar 

  • Williamson, M. (1996). Biological invasions. London: Chapman and Hall.

    Google Scholar 

Download references

Acknowledgments

The authors thank CREST for the use of SA Knowledgebase, and the former database manager, Derick van Niekerk, for running the keyword searches. Acknowledgements are due to the Director (Steven Chown) and Deputy Director (David Richardson) of the CIB for their assistance in identifying the sex of some of the authors in the dataset, and in identifying relevant keywords. David Richardson is also thanked for highlighting the need to control for the time available for accumulation of citations, and his and Dan Simberloff’s insightful comments on an earlier draft of this paper, as well as the comments of two anonymous reviewers, are much appreciated. Research assistance for this project was funded from the first author’s CIB core team member grant. Finally, we are highly indebted to Charline Mouton, Marion van Dorssen and Christopher Mechnig for their assistance in collecting data for this project.

Author information

Authors and Affiliations

  1. Department of Sociology and Social Anthropology and DST-NRF Centre of Excellence for Invasion Biology, Stellenbosch University, Private Bag x1, Matieland, South Africa

    Heidi Prozesky

  2. Centre for Research on Evaluation, Science and Technology, Stellenbosch University, Private Bag x1, Matieland, South Africa

    Nelius Boshoff

Authors
  1. Heidi Prozesky
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nelius Boshoff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heidi Prozesky.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Prozesky, H., Boshoff, N. Bibliometrics as a tool for measuring gender-specific research performance: an example from South African invasion ecology. Scientometrics 90, 383–406 (2012). https://doi.org/10.1007/s11192-011-0478-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-011-0478-7

Keywords

Search

Navigation