Abstract
This research aims to evaluate the quality of the literature search strategy of recently published bibliometric studies in the field of Business and Economics. The search strategy is evaluated from two perspectives, i.e., reporting quality and the ability of search query to find relevant literature. Results showed that selected bibliometric studies did not report their search strategies effectively. Particularly, keyword selection, complete search query and search space are not reported adequately. Results also revealed that the search query quality was not good in most cases, especially in selecting the appropriate synonyms, applying the Boolean operators, and applying the appropriate search space. Further, this research recommended a “crisscross” strategy to extract the relevant literature. It is suggested that future studies can increase the search query quality by adopting the suggested framework.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We started this article in June 2022 and completed in January 2023. Since, the project was started in mid of 2022, therefore the recent year at that time was taken as 2021.
References
Alharbi, A., & Stevenson, M. (2020). Refining boolean queries to identify relevant studies for systematic review updates. Journal of the American Medical Informatics Association, 27(11), 1658–1666. https://doi.org/10.1093/jamia/ocaa148
Ali, N. B., & Usman, M. (2018). Reliability of search in systematic reviews: Towards a quality assessment framework for the automated-search strategy. Information and Software Technology, 99, 133–147. https://doi.org/10.1016/j.infsof.2018.02.002
Almubarak, D., Pandis, N., Cobourne, M. T., & Seehra, J. (2021). Reporting of the methodological quality of search strategies in orthodontic quantitative systematic reviews. European Journal of Orthodontics, 43(5), 551–556. https://doi.org/10.1093/ejo/cjaa078
Ayasse, M. T., Ahmed, A., Espinosa, M. L., Walker, C. J., Yousaf, M., Thyssen, J. P., & Silverberg, J. I. (2021). What are the highest yielding search strategy terms for systematic reviews in atopic dermatitis? A systematic review. Archives of Dermatological Research, 313(9), 737–750. https://doi.org/10.1007/s00403-020-02165-z
Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., & Lim, W. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of Business Research, 133, 285–296. https://doi.org/10.1016/j.jbusres.2021.04.070
Ellegaard, O., & Wallin, J. A. (2015). The bibliometric analysis of scholarly production: How great is the impact? Scientometrics, 105(3), 1809–1831. https://doi.org/10.1007/s11192-015-1645-z
Eskrootchi, R., Mohammadi, A. S., Panahi, S., & Zahedi, R. (2020). Librarians’ participation in the systematic reviews published by iranian researchers and its impact on the quality of reporting search strategy. Evidence Based Library and Information Practice, 15(2), 69–84. https://doi.org/10.18438/eblip29609
Faggion, C. M., Wu, Y. C., Tu, Y. K., & Wasiak, J. (2016). Quality of search strategies reported in systematic reviews published in stereotactic radiosurgery. British Journal of Radiology, 89(1062), 1–7. https://doi.org/10.1259/bjr.20150878
Franco, J. V. A., Garrote, V. L., Escobar Liquitay, C. M., & Vietto, V. (2018). Identification of problems in search strategies in cochrane reviews. Research Synthesis Methods, 9(3), 408–416. https://doi.org/10.1002/jrsm.1302
Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
Grindlay, D. J. C., & Karantana, A. (2018). Putting the ‘systematic’ into searching – tips and resources for search strategies in systematic reviews. Journal of Hand Surgery: European, 43(6), 674–678. https://doi.org/10.1177/1753193418778978
Hausner, E., Waffenschmidt, S., Kaiser, T., & Simon, M. (2012). Routine development of objectively derived search strategies. Systematic Reviews, 1(1), 1–10. https://doi.org/10.1186/2046-4053-1-19
Hausner, E., Guddat, C., Hermanns, T., Lampert, U., & Waffenschmidt, S. (2015). Development of search strategies for systematic reviews: Validation showed the noninferiority of the objective approach. Journal of Clinical Epidemiology, 68(2), 191–199. https://doi.org/10.1016/j.jclinepi.2014.09.016
Herrström, K., Larsson, S., Einberg, E. L., Nilsson, M., Blomqvist, K., & Garmy, P. (2020). Assessment of search strategies in literature-review-based candidate theses within a nursing program. Advances in Medical Education and Practice, 11, 71–77. https://doi.org/10.2147/AMEP.S227547
Liu, B., Lu, X., & Culpepper, J. S. (2021). Strong natural language query generation. Information Retrieval Journal, 24(4–5), 322–346. https://doi.org/10.1007/s10791-021-09395-3
Marler, J. L., Curry, K. Y., & Suda, K. J. (2014). Assessing the quality of database search strategies utilized by meta-analyses that evaluate infectious disease drug therapy. Therapeutic Innovation and Regulatory Science, 48(3), 336–340. https://doi.org/10.1177/2168479013516774
Mcgowan, J., Sampson, M., Salzwedel, D. M., Cogo, E., Foerster, V., & Lefebvre, C. (2016). GUIDELINE STATEMENT PRESS peer review of electronic search strategies : 2015 guideline statement. Journal of Clinical Epidemiology, 75, 40–46. https://doi.org/10.1016/j.jclinepi.2016.01.021
Molina-Collado, A., Salgado-Sequeiros, J., Gómez-Rico, M., Aranda García, E., & de Maeyer, P. (2021). Key themes in consumer financial services research from 2000 to 2020: A bibliometric and science mapping analysis. International Journal of Bank Marketing, 39(7), 1446–1478. https://doi.org/10.1108/IJBM-01-2021-0043
Mourão, E., Pimentel, J. F., Murta, L., Kalinowski, M., Mendes, E., & Wohlin, C. (2020). On the performance of hybrid search strategies for systematic literature reviews in software engineering. Information and Software Technology. https://doi.org/10.1016/j.infsof.2020.106294
Paul, J., & Criado, A. R. (2020). The art of writing literature review: What do we know and what do we need to know? International Business Review, 29(4), 101717. https://doi.org/10.1016/j.ibusrev.2020.101717
Peltonen, L. M., Tommila, M., & Moen, H. (2020). Exploring unsupervised query paraphrasing to identify relevant search phrases for a literature review. Studies in Health Technology and Informatics, 272, 429–432. https://doi.org/10.3233/SHTI200587
Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015). Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. Journal of Clinical Epidemiology, 68(6), 617–626. https://doi.org/10.1016/j.jclinepi.2014.11.025
Roundtree, A. K., Kallen, M. A., Lopez-Olivo, M. A., Kimmel, B., Skidmore, B., Ortiz, Z., et al. (2009). Poor reporting of search strategy and conflict of interest in over 250 narrative and systematic reviews of two biologic agents in arthritis: A systematic review. Journal of Clinical Epidemiology, 62(2), 128–137. https://doi.org/10.1016/j.jclinepi.2008.08.003
Rousseau, S., & Rousseau, R. (2021). Bibliometric techniques and their use in business and economics research. Journal of Economic Surveys, 35(5), 1428–1451. https://doi.org/10.1111/joes.12415
Scells, H., Zuccon, G., & Koopman, B. (2021). A comparison of automatic Boolean query formulation for systematic reviews. Information Retrieval Journal, 24(1), 3–28. https://doi.org/10.1007/s10791-020-09381-1
Van Eck, N. J., & Waltman, L. (2017). Citation-based clustering of publications using CitNetExplorer and VOSviewer. Scientometrics, 111(2), 1053–1070. https://doi.org/10.1007/s11192-017-2300-7
Wohlin, C., Mendes, E., Felizardo, K. R., & Kalinowski, M. (2020). Guidelines for the search strategy to update systematic literature reviews in software engineering. Information and Software Technology, 127, 106366. https://doi.org/10.1016/j.infsof.2020.106366
Yoshii, A., Plaut, D. A., McGraw, K. A., Anderson, M. J., & Wellik, K. E. (2009). Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association, 97(1), 21–29. https://doi.org/10.3163/1536-5050.97.1.004
Zupic, I., & Čater, T. (2015). Bibliometric methods in management and organization. Organizational Research Methods, 18(3), 429–472. https://doi.org/10.1177/1094428114562629
Li, H., Scells, H., & Zuccon, G. (2020). Systematic Review Automation Tools for End-to-End Query Formulation. In: SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc. (pp. 2141–2144) https://doi.org/10.1145/3397271.3401402
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Farooq, U., Nasir, A. & Khan, K.I. An assessment of the quality of the search strategy: a case of bibliometric studies published in business and economics. Scientometrics 128, 4855–4874 (2023). https://doi.org/10.1007/s11192-023-04765-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-023-04765-8