Is it better or just the same? Article identification strategies impact bibliometric assessments

Summary

This study demonstrates that the choice of search strategy for article identification has an impact on evaluation and policy analysis of research areas. We have assessed the scientific production in two areas at one research institution during a ten-year period. We explore the recall and precision of three article identification strategies: journal classifications, keywords and authors. Our results show that the different search strategies have varying recall (0.38-1.00) and precision (0.50-1.00). In conclusion, uncritical analysis based on rudimentary article identification strategies may lead to misinterpretation of the development of research areas, and thus provide incorrect data for decision-making.