Skip to main content
Springer Nature Link
Log in

Multiparametric characterization of scientometric performance profiles assisted by neural networks: a study of Mexican higher education institutions

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Development of accurate systems to assess academic research performance is an essential topic in national science agendas around the world. Providing quantitative elements such as scientometric rankings and indicators have contributed to measure prestige and excellence of universities, but more sophisticated computational tools are seldom exploited. We compare the evolution of Mexican scientific production in Scopus and the Web of Science as well as Mexico’s scientific productivity in relation to the growth of the National Researchers System of Mexico is analyzed. As a main analysis tool we introduce an artificial intelligence procedure based on self-organizing neural networks. The neural network technique proves to be a worthy scientometric data mining and visualization tool which automatically carries out multiparametric scientometric characterizations of the production profiles of the 50 most productive Mexican Higher Education Institutions (in Scopus database). With this procedure we automatically identify and visually depict clusters of institutions that share similar bibliometric profiles in bidimensional maps. Four perspectives were represented in scientometric maps: productivity, impact, expected visibility and excellence. Since each cluster of institutions represents a bibliometric pattern of institutional performance, the neural network helps locate various bibliometric profiles of academic production, and the identification of groups of institutions which have similar patterns of performance. Also, scientometric maps allow for the identification of atypical behaviors (outliers) which are difficult to identify with classical tools, since they outstand not because of a disparate value in just one variable, but due to an uncommon combination of a set of indicators values.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Abramo, G., Cicero, T., & D’Angelo, C. A. (2013). National peer-review research assessment exercises for the hard sciences can be a complete waste of money: The Italian case. Scientometrics, 95(1), 311–324.

    Article  Google Scholar 

  • Aguillo, I. F., Bar-Ilan, J., Levene, M., & Ortega, J. L. (2010). Comparing university rankings. Scientometrics, 85(1), 243–256.

    Article  Google Scholar 

  • Aguillo, I. F., Granadino, B., & Llamas, G. (2005). Web positioning of the university system in Latin America. Interciencia, 30(12), 735–738.

    Google Scholar 

  • Allen, N., & Heath, O. (2013). Reputations and research quality in british political science: The importance of journal and publisher rankings in the 2008 RAE. British Journal of Politics & International Relations, 15(1), 147–162.

    Article  Google Scholar 

  • Arencibia-Jorge, R., & de Moya-Anegon, F. (2010). Challenges in the study of Cuban scientific output. Scientometrics, 83(3), 723–737.

    Article  Google Scholar 

  • Arvanitis, R., Russell, J. M., & Rosas, A. M. (1996). Experiences with the national citation reports database for measuring national performance: The case of Mexico. Scientometrics, 35(2), 247–255.

    Article  Google Scholar 

  • Belkhodja, O., & Landry, R. (2007). The Triple-helix collaboration: Why do researchers collaborate with industry and the government? What are the factors that influence the perceived barriers? Scientometrics, 70(2), 301–332.

    Article  Google Scholar 

  • Belter, C. W. (2013). A bibliometric analysis of NOAA’s Office of Ocean Exploration and Research. Scientometrics, 95(2), 629–644.

    Article  Google Scholar 

  • Bengoetxea, E., & Buela-Casal, G. (2013). The new multidimensional and user-driven higher education ranking concept of the European Union. International Journal of Clinical and Health Psychology, 13(1), 67–73.

    Article  Google Scholar 

  • Benito, M., & Romera, R. (2011). Improving quality assessment of composite indicators in university rankings: A case study of French and German universities of excellence. Scientometrics, 89(1), 153–176.

    Article  Google Scholar 

  • Billaut, J. C., Bouyssou, D., & Vincke, P. (2010). Should you believe in the Shanghai ranking? Scientometrics, 84(1), 237–263.

    Article  Google Scholar 

  • Bornmann, L., & Leydesdorff, L. (2013). Macro-indicators of citation impacts of six prolific countries: InCites data and the statistical significance of trends. PLoS ONE, 8(2), e56768.

    Article  Google Scholar 

  • Bornmann, L., Moya-Anegón, F., & Leydesdorff, L. (2012). The new excellence indicator in the World Report of the SCImago Institutions Rankings 2011. Journal of Informetrics, 6(2), 333–335.

    Article  Google Scholar 

  • Caputo, C., Requena, J., & Vargas, D. (2012). Life sciences research in Venezuela. Scientometrics, 90(3), 781–805.

    Article  Google Scholar 

  • Collazo-Reyes, F., Luna-Morales, M. E., Russell, J. M., & Perez-Angon, M. A. (2008). Publication and citation patterns of Latin American and Caribbean journals in the SCI and SSCI from 1995 to 2004. Scientometrics, 75(1), 145–161.

    Article  Google Scholar 

  • Collazo-Reyes, F., Luna-Morales, M. E., Russell, J. M., & Perez-Angon, M. A. (2010). Enriching knowledge production patterns of Mexican physics in particles and fields. Scientometrics, 85(3), 791–802.

    Article  Google Scholar 

  • CONACyT. (2012). Atlas de la Ciencia Mexicana. Mexico D.F: Academia Mexicana de Ciencias.

    Google Scholar 

  • CONACyT. (2013). Resultados Anteriores del Sistema Nacionales de Investigadores (SNI). http://conacyt.gob.mx/index.php/resultados-anteriores.

  • Dunn, J. C. (1973). A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. Journal of Cybernetics, 3(3), 32–57.

    Article  MathSciNet  MATH  Google Scholar 

  • Egghe, L. (2012). Averages of ratios compared to ratios of averages: Mathematical results. Journal of Informetrics, 6(2), 307–317.

    Article  Google Scholar 

  • Foro Consultivo Científico y Tecnológico. (2011). Ranking de Producción Científica Mexicana. http://www.foroconsultivo.org.mx/libros_editados/ranking_por_institucion_2011.pdf.

  • Gomez-Nunez, A. J., Vargas-Quesada, B., de Moya-Anegon, F., & Glanzel, W. (2011). Improving SCImago Journal and Country Rank (SJR) subject classification through reference analysis. Scientometrics, 89(3), 741–758.

    Article  Google Scholar 

  • Guzmán, M. V., Carrillo, H., Jiménez, J. L., & Villaseñor, E. A. (2010). Bioinformetric studies on TB vaccines research. In: N. M. Nor, A. Acosta & M. E. Sarmiento (eds.), The art and science of Tuberculosis vaccine development (pp. 425–441). Oxford: Oxford University Press.

    Google Scholar 

  • Helene, A. F., & Ribeiro, P. L. (2011). Brazilian scientific production, financial support, established investigators and doctoral graduates. Scientometrics, 89(2), 677–686.

    Article  Google Scholar 

  • Huang, M. H. (2012). Exploring the h-index at the institutional level: A practical application in world university rankings. Online Information Review, 36(4), 534–547.

    Article  Google Scholar 

  • Ioannidis, J. P., Patsopoulos, N. A., Kavvoura, F. K., Tatsioni, A., Evangelou, E., Kouri, I., et al. (2007). International ranking systems for universities and institutions: A critical appraisal. BMC Medicine, 5(30), 1.

    Google Scholar 

  • Kohonen, T. (2001). Self-organizing maps (3rd ed.). Berlin: Springer.

    Book  MATH  Google Scholar 

  • Kohonen, T. (2013). Essentials of the self-organizing map. Neural Networks, 37, 52–65.

    Article  Google Scholar 

  • Krauskopf, M., Krauskopf, E., & Mendez, B. (2007). Low awareness of the link between science and innovation affects public policies in developing countries: The Chilean case. Scientometrics, 72(1), 93–103.

    Article  Google Scholar 

  • Krishna, D., Mohan, S. R., Murthy, B. S. N., & Rao, A. R. (2002). Performance evaluation of public research institutes using principal component analysis. Journal of Scientific & Industrial Research, 61(11), 940–947.

    Google Scholar 

  • Kurzydlowski, K. J. (2003). Materials research in Poland. In W. Lojkowski & J. R. Blizzard (Eds.), Interfacial effects and novel properties of nanomaterials (pp. 349–356). Zurich-Uetikon: Trans Tech Publications Ltd.

    Google Scholar 

  • Lena, M. (1997). Scientific productivity in environmental psychology in Mexico—A bibliometric analysis. Environment and Behavior, 29(2), 169–197.

    Article  Google Scholar 

  • Leydesdorff, L., & Meyer, M. (2007). The scientometrics of a Triple Helix of university–industry–government relations (introduction to the topical issue). Scientometrics, 70(2), 207–222.

    Article  Google Scholar 

  • Licea de Arenas, J., Castanos-Lomnitz, H., & Arenas-Licea, J. (2002). Significant Mexican research in the health sciences: A bibliometric analysis. Scientometrics, 53(1), 39–48.

    Article  Google Scholar 

  • Licea de Arenas, J., Valles, J., & Arenas, M. (2000). Educational research in Mexico: Socio-demographic and visibility issues. Educational Research, 42(1), 85–90.

    Article  Google Scholar 

  • Luna-Morales, M. E. (2012). International scientific collaboration and recognition of Mexican science from 1980 to 2004. Investigacion Bibliotecologica, 26(57), 103–129.

    Google Scholar 

  • Macias-Chapula, C. A., Mendoza-Guerrero, J. A., Rodea-Castro, I. P., & Gutierrez-Carrasco, A. (2007). Institutional health research collaboration in Mexico: A Bibliometric study. In: Proceedings of the 11th international conference of the international society for scientometrics and informetrics (pp. 894–895). Madrid: ISSI.

  • Miguel, S., Chinchilla-Rodríguez, Z., & Moya-Anegón, F. (2011). Open access and Scopus: A new approach to scientific from the standpoint of access. Journal of the American Society for Information Science and Technology, 62(6), 1130–1145.

    Article  Google Scholar 

  • Miguel, S., Moya-Anegon, F., & Herrero-Solana, V. (2010). The impact of the socio-economic crisis of 2001 on the scientific system of Argentina from the scientometric perspective. Scientometrics, 85(2), 495–507.

    Article  Google Scholar 

  • Moya-Anegón, F., Herrero-Solana, V., & Jiménez-Contreras, E. (2006). A connectionist and multivariate approach to science maps: The SOM, clustering and MDS applied to library and information science research. Journal of Information Science, 32(1), 63–77.

    Article  Google Scholar 

  • Mryglod, O., Kenna, R., Holovatch, Y., & Berche, B. (2013). Absolute and specific measures of research group excellence. Scientometrics, 95(1), 115–127.

    Article  Google Scholar 

  • Noyons, C. M. (2005). Science maps within a science policy context. In: H. F. Moed, W. Glänzel & U. Schmoch (eds.), Handbook of quantitative science and technology research (pp. 237–255). Springer: Netherlands.

    Chapter  Google Scholar 

  • Polanco, X., François, C., & Lamirel, J. C. (2001). Using artificial neural networks for mapping of science and technology: A multi-self-organizing-maps approach. Scientometrics, 51(1), 267–292.

    Article  Google Scholar 

  • Rehn, C., & Kronman, U. (2008). Bibliometric handbook for Karolinska Institutet. Estocolmo: Karolinska Institutet University Library.

    Google Scholar 

  • SCImago Research Group. (2013). SIR Iber 2013. http://www.scimagoir.com/pdf/SIR%20Iber%202013.pdf.

  • Sierra-Flores, M. M., & Barnard, J. M. R. (2009). The most productive research groups of the National Autonomous University of Mexico (UNAM) in the area of physics, 1990–1999. Investigacion Bibliotecologica, 23(48), 127–155.

    Google Scholar 

  • Sierra-Flores, M. M., Guzman, M. V., Raga, A. C., & Perez, I. (2009). The productivity of Mexican astronomers in the field of outflows from young stars. Scientometrics, 81(3), 765–777.

    Article  Google Scholar 

  • Skupin, A., Biberstine, J. R., & Börner, K. (2013). Visualizing the topical structure of the medical sciences: A self-organizing map approach. PLoS ONE, 8(3), e58779.

    Article  Google Scholar 

  • Sotolongo-Aguilar, G., Guzmán-Sánchez, M. V., & Carrillo-Calvet, H. (2002). ViBlioSOM: Visualización de información bibliométrica mediante el mapeo autoorganizado. Revista Española de Documentación Científica, 25(4), 477–484.

    Article  Google Scholar 

  • Sotolongo-Aguilar, G., Guzmán-Sánchez, M. V., Saavedra-Fernández, O., Carrillo-Calvet, H. A. (2001). Mining informetric data with self-organizing maps. In Proceedings of the 8th international society for scientometrics and informetrics (pp. 665–673). Sydney: BIRG.

  • Statzner, B., & Resh, V. H. (2010). Negative changes in the scientific publication process in ecology: potential causes and consequences. Freshwater Biology, 55(12), 2639–2653.

    Article  Google Scholar 

  • Thomson Reuters. (2015). http://incites.isiknowledge.com/common/help/h_glossary.html.

  • Torres-Salinas, D., Moreno-Torres, J. G., Delgado-Lopez-Cozar, E., & Herrera, F. (2011). A methodology for institution-field ranking based on a bidimensional analysis: The IFQ(2)A index. Scientometrics, 88(3), 771–786.

    Article  Google Scholar 

  • Ultsch, A., & Mörchen, F. (2005). ESOM-maps: Tools for clustering, visualization, and classification with emergent SOM. Technical report. 46. Department of Mathematics and Computer Science, University of Marburg, Germany.

  • UNAM. (2012). Estudio Comparativo de Universidades Mexicanas. http://www.ecum.unam.mx/.

  • Vanclay, J. K., & Bornmann, L. (2012). Metrics to evaluate research performance in academic institutions: A critique of ERA 2010 as applied in forestry and the indirect H-2 index as a possible alternative. Scientometrics, 91(3), 751–771.

    Article  Google Scholar 

  • Vesanto, J. (1999). SOM-based data visualization methods. Intelligent Data Analysis, 3(2), 111–126.

    Article  MATH  Google Scholar 

  • Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. (2011). Towards a new crown indicator: An empirical analysis. Scientometrics, 87(3), 467–481.

    Article  Google Scholar 

  • Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62(1), 117–131.

    Article  Google Scholar 

  • Zell, D. (2005). Pressure for relevancy at top-tier business schools. Journal of Management Inquiry, 14(3), 271–274.

    Article  MathSciNet  Google Scholar 

  • Zhao, S. X., Zheng, L. C., & Cai, Z. C. (2009). Evaluation of science and technology innovation ability of colleges and universities in China. In International conference on management of tech-nology, pp. 220–225.

Download references

Acknowledgements

This research was partially supported by the Proyecto CITMA-CONACyT (B330.166) and the Empresa de Tecnologías Inteligentes y Modelación de Sistemas S.A. de C.V. The authors acknowledge the collaboration of José Luis Jiménez Andrade (UNAM, Mexico), and of Dr. Félix de Moya Anegón (CSIC, Spain) for the data support given from SCImago Institutions Rankings.

Author information

Authors and Affiliations

  1. Center of Research and Innovation in Information and Communication Technologies INFOTEC, Circuito Tecnopolo Sur 112, Mexico, Aguascalientes, Mexico

    Elio Atenógenes Villaseñor

  2. Empresa de Tecnologías de la Información, La Habana, Cuba

    Ricardo Arencibia-Jorge

  3. Laboratory of Nonlinear Dynamics, Faculty of Sciences and Center of Complexity Sciences, National Autonomous University of Mexico, Mexico City, D.F., Mexico

    Humberto Carrillo-Calvet

Authors
  1. Elio Atenógenes Villaseñor
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ricardo Arencibia-Jorge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Humberto Carrillo-Calvet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Humberto Carrillo-Calvet.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Villaseñor, E.A., Arencibia-Jorge, R. & Carrillo-Calvet, H. Multiparametric characterization of scientometric performance profiles assisted by neural networks: a study of Mexican higher education institutions. Scientometrics 110, 77–104 (2017). https://doi.org/10.1007/s11192-016-2166-0

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-016-2166-0

Keywords

Mathematics Subject Classification

JEL