Abstract
The aim of the present study is to provide an overview of the research activity, in terms of productivity and impact, of the Italian public universities and their academic staff in the engineering sector (EngSec), that is about seven thousand researchers. Beyond the increased bibliometric performance of the considered research area in the last decade, attention will be focused on the possible feedback on publication habits induced by regulations which progressively introduced, in the research world, strong competitiveness elements, such as evaluation exercises, covering both the single researcher career and the funding system. The effects of this feedback are also explored from the point of view of the self-citation phenomenon, which is said to increasingly affect the research world at large. The EngSec is analyzed both as a whole and decomposed in cultural areas to give a more comprehensive view of this important cultural segment of the academia. In 10 years, the productivity of the sector in term of Scopus documents is roughly doubled, with a consistent shift from conference papers to journal articles. The impact also showed a similar trend with self-citations exposing an evident peak in the year 2013.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Abbreviations
- ANVUR:
-
IT National Agency for the Evaluation of the University and Research System
- ASN:
-
National Scientific Qualification
- Cineca:
-
Interuniversity Consortium High Performance Systems
- CIRV:
-
Committee for the research evaluation
- CUN:
-
IT National University Council
- ERA:
-
Excellence in Research for Australia
- FCT:
-
Evaluation of the Portuguese Foundation for Science and Technology
- MIUR:
-
IT Ministry of Education, Universities and Research
- RAE:
-
UK Research Assessment Exercise
- REF:
-
UK Research Excellence Framework
- VQR:
-
Research Quality Assessment
- VTR:
-
Three year Research Evaluation
References
Abramo, G., & D’Angelo, C. A. (2011). National-scale research performance assessment at the individual level. Scientometrics, 86(2), 347–364. https://doi.org/10.1007/s11192-010-0297-2.
Abramo, G., D’Angelo, C. A., & Pugini, F. (2008). The measurement of Italian universities’ research productivity by a non parametric-bibliometric methodology. Scientometrics, 76(2), 225–244. https://doi.org/10.1007/s11192-007-1942-2.
Aksnes, D. W. (2003). A macro study of self-citation. Scientometrics, 56(2), 235–246. https://doi.org/10.1023/A:1021919228368.
Ancaiani, A., Anfossi, A. F., Barbara, A., Benedetto, S., Blasi, B., Carletti, V., et al. (2015). Evaluating scientific research in Italy: The 2004–10 research evaluation exercise. Research Evaluation, 24(3), 242–255. https://doi.org/10.1093/reseval/rvv008.
ANVUR. (2011). VQR 2004–2010: Riferimenti normativi (Reference legislation). http://www.anvur.org/index.php?option=com_content&view=article&id=122&Itemid=305&lang=it. Accessed March 02, 2018.
Baier-Fuentes, H., Merigó, J. M., Amorós, J. E., & Gaviria-Marín, M. (2018). International entrepreneurship: A bibliometric overview. International Entrepreneurship and Management Journal. https://doi.org/10.1007/s11365-017-0487-y.
Cineca. (2017). Ministerial database of Italian academics. http://cercauniversita.cineca.it/php5/docenti/cerca.php. Accessed March 02, 2018.
Costas, R., van Leeuwen, T. N., & Bordons, M. (2010). Self-citations at the meso and individual levels: Effects of different calculation methods. Scientometrics, 82(3), 517–537. https://doi.org/10.1007/s11192-010-0187-7.
C.U.N. (2009). DOCUMENTO INTRODUTTIVO SU REVISIONE SSD (in Italian). https://www.cun.it/uploads/storico/documento_ssd.pdf. Accessed June 02, 2018.
Drott, M. C. (1995). Reexamining the role of conference papers in scholarly communication. Journal of the American Society for Information Science, 46(4), 299–305. https://doi.org/10.1002/(SICI)1097-4571(199505)46:4%3c299:AID-ASI6%3e3.0.CO;2-0.
Franceschini, F., & Maisano, D. (2017). Critical remarks on the Italian research assessment exercise VQR 2011–2014. Journal of Informetrics, 11, 337–357. https://doi.org/10.1016/j.joi.2017.02.005.
Garfield, E. (1979). Is citation analysis a legitimate evaluation tool? Scientometrics, 1(4), 359–375. https://doi.org/10.1007/BF02019306.
Glänzel, W. (2004). Towards a model for diachronous and synchronous citation analyses. Scientometrics, 60(3), 511–522. https://doi.org/10.1023/B:SCIE.0000034391.06240.2a.
Glänzel, W., Thijs, B., & Schlemmer, B. (2004). A bibliometric approach to the role of author self-citations in scientific communication. Scientometrics, 59(1), 63–77. https://doi.org/10.1023/B:SCIE.0000013299.38210.74.
He, Y., & Guan, J. C. (2008). Contribution of Chinese publications in computer science: A case study on LNCS. Scientometrics, 75(3), 519–534. https://doi.org/10.1007/s11192-007-1781-1.
Ibáñez, A., Larrañaga, P., & Bielza, C. (2013). Cluster methods for assessing research performance: Exploring Spanish computer science. Scientometrics, 97(3), 571–600. https://doi.org/10.1007/s11192-013-0985-9.
Lawani, S. M. (1982). On the heterogeneity and classification of author self-citations. Journal of the American Society for Information Science and Technology, 33(5), 281–284. https://doi.org/10.1002/asi.4630330506.
López-Illescas, C., de Moya-Anegón, F., & Moed, H. F. (2008). The actual citation impact of European oncological research. European Journal of Cancer, 44(2), 228–236. https://doi.org/10.1016/j.ejca.2007.10.020.
Marzolla, M. (2016). Assessing evaluation procedures for individual researchers: The case of the Italian National Scientific Qualification. Journal of Informetrics, 10, 408–438. https://doi.org/10.1016/j.joi.2016.01.009.
May, K. O. (1967). Abuses of citation indexing. Science, 157, 890–891. https://doi.org/10.1126/science.156.3777.890-a.
MIUR. (2010). ASN, National Scientific Qualification: Riferimenti normativi (Reference legislation). http://abilitazione.miur.it/public/normativa.php?sersel=1&lang=eng. Accessed March 02, 2018.
MIUR. (2015). Decreto Ministeriale 30 ottobre 2015 n. 855, Rideterminazione dei macrosettori e dei settori concorsuali (annex D). http://attiministeriali.miur.it/media/265763/allegato_d.pdf. Accessed March 02, 2018.
MIUR. (2016). Decreto Ministeriale 7 ottobre 2016 n. 120, Regolamento recante criteri e parametri per la valutazione dei candidati ai fini dell’attribuzione dell’abilitazione scientifica nazionale…. http://attiministeriali.miur.it/media/281128/dm_120.pdf. Accessed March 02, 2018.
Moed, H. F., Burger, W. J. M., Frankfort, J. G., & Van Raan, A. F. J. (1985). The use of bibliometric data for the measurement of university research performance. Research Policy, 14(3), 131–149. https://doi.org/10.1016/0048-7333(85)90012-5.
Pichappan, P., & Sarasvady, S. (2002). The other side of the coin: The intricacies of author self-citations. Scientometrics, 54(2), 285–290. https://doi.org/10.1023/A:1016070029935.
Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253–280. https://doi.org/10.1007/s11192-007-1722-z.
Tagliacozzo, R. (1977). Self-citations in scientific literature. Journal of Documentation, 33(4), 251–265. https://doi.org/10.1108/eb026644.
Van Raan, A. F. J. (2008). Self-citation as an impact-reinforcing mechanism in the science system. Journal of the American Society for Information Science and Technology, 59(10), 1631–1643. https://doi.org/10.1002/asi.20868.
Wang, J. (2013). Citation time window choice for research impact evaluation. Scientometrics, 94(3), 851–872. https://doi.org/10.1007/s11192-012-0775-9.
Zijlstra, H., & McCullough, R. (2016). CiteScore: A new metric to help you track journal performance and make decisions. Pub online, https://www.elsevier.com/editors-update/story/journal-metrics/citescore-a-new-metric-to-help-you-choose-the-right-journal. Accessed 12 Sept 2018.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Scarpa, F., Bianco, V. & Tagliafico, L.A. The impact of the national assessment exercises on self-citation rate and publication venue: an empirical investigation on the engineering academic sector in Italy. Scientometrics 117, 997–1022 (2018). https://doi.org/10.1007/s11192-018-2913-5
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-018-2913-5