Abstract
There are increasing demands on universities to operate transparently with regard how resources are being used and targets met, putting them under growing pressure to clarify their position relative to other universities at national and international level. But there are several challenges associated with establishing their relative positioning. The first issue is concerned with measurement or how to capture data that are relevant, pertinent and fit-for-purpose. Second, universities must obtain an overall indicator or a means of ordering that helps synthesize the different indicators; and, third, they must decide how to weight them. Those university rankings that attempt to address these questions are met with a degree of criticism for being subjective or inconsistent in their quantification of the indicators. The aim of the present work is to develop a procedure for synthesizing all of the indicators relating to the objective measurement of university R&D and innovation into a single or summary concept. In other words, to establish a procedure that does not require subjective criteria and that can be applied for both absolute and relativized indicators. This approach makes a dual contribution. First, a specific application, in this particular case for the case of the Spanish university system, will be created, to obtain a synthetic indicator for research activity. This will enable us to achieve an R&D and innovation ranking for Spanish universities. Second, the work makes a methodological contribution by using a new technique for synthesizing this type of indicator, namely Partial Least Squares (PLS).
Similar content being viewed by others
Notes
The missing values are imputated using the scores predicted by a series of regressions in which each incomplete variable is regressed on the remaining variables for a given case. In the next stage (maximization), the entire set of imputated data is subjected to estimation via the maximum likelihood method. Each of these two steps is repeated in sequence until a stable solution is achieved.
References
Aghion, P., Dewatripont, M., Hoxby, C., Mas-Colell, A., & Sapir, A. (2010). The governance and performance of universities: Evidence from Europe and the US. Economic Policy, 25(61), 7–59.
Altbach, P. G., Reisberg, L., & Rumbley, L. E. (2010). Tracking a global academic revolution. Change: The Magazine of Higher Education, 42(2), 30–39.
Bagozzi, R. P. (1980). Causal models in marketing. New York: Wiley.
Bentler, P. M. (1980). Multivariate analysis with latent variables: Causal models. Annual Review of Psychology, 31, 419–456.
Buela-Casal, G., Bermúdez-Sánchez, M. P., Sierra, J. C., Quevedo-Blasco, R., & Castro, A. (2010). Ranking de 2009 en investigación de las universidades públicas españolas. Psicothema, 22(2), 171–179.
Buela-Casal, G., Gutiérrez-Martínez, O., Bermúdez-Sánchez, M. P., & Vadillo-Muñoz, O. (2007). Comparative study of international academic rankings of universities. Scientometrics, 71(3), 349–365.
Carayannis, E. G., & Campbell, D. F. J. (2011). Open innovation diplomacy and a 21st century fractal research, education and innovation (FREIE) ecosystem: Building on the quadruple and quintuple helix innovation concepts and the “Mode 3” knowledge production system. Journal of the Knowledge Economy, 2(3), 257–326.
Chen, K., & Liao, P. (2012). A comparative study on world university rankings: A bibliometric survey. Scientometrics, 92(1), 89–103.
Chin, W. W. (1998a). Issues and opinion on structural equation modelling. MIS Quarterly, 22(1), 8–15.
Chin, W. W. (1998b). The partial least squares approach to structural equation modelling. In G. A. Marcoulides (Ed.), Modern methods for business research (pp. 295–336). Mahwah: Lawrence Erlbaum Associates, Publisher.
Del Barrio-García, S. & Luque-Martínez, T. (2012). Análisis de ecuaciones estructurales. In Luque-Martínez, T. Técnicas de análisis de datos en investigación de mercados. Madrid. Ed. Pirámide.
Docampo, D. (2011). On using the Shanghai ranking to assess the research performance of university systems. Scientometrics, 86(1), 77–92.
Docampo, D. (2012). Adjusted sum of institutional scores as an indicator of the presence of university systems in the ARWU ranking. Scientometrics, 90(1), 701–713.
Docampo, D., Herrera, F., Luque-Martínez, T., & Torres-Salinas, D. (2012). Efecto de la agregación de universidades españolas en el ranking de Shanghai (ARWU): caso de las comunidades autónomas y los campus de excelencia”. El profesional de la información, 21(4), 428–432.
Etzkowitz, H., & Leydesdorff, L. (1998). The endless transition: A ‘‘triple helix’’ of university–industry–government relations. Minerva, 36, 203–208.
Filippo, D., Sanz-Casado, E., Urbano Salido, C., Ardanuy, J., & Gómez-Caridad, J. (2011). El papel de las bases de datos institucionales en el análisis de la actividad científica de las universidades. Revista Española de Documentación Científica, 34(2), 165–189.
Fornell, C., & Cha, J. (1994). Partial least squares. In Richard P. Bagozzi (Ed.), Advanced methods of marketing research (pp. 52–78). Cambridge MA: Blackwell.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39–50.
García, J. A., Rodríguez-Sánchez, R., Fdez-Valdivia, J., Robinson-García, N., & Torres-Salinas, D. (2012). Mapping academic institutions according to their journal publication profile: Spanish universities as a case study. Journal of the American Society for Information Science and Technology, 63(11), 2328–2340.
Hazelkorn, E. (2011). Rankings and the reshaping of higher education. New York: Palgrave Macmillan.
Ishikawa, M. (2009). University rankings, global models, and emerging hegemony critical analysis from Japan. Journal of Studies in International Education, 13(2), 159–173.
Jiménez-Contreras, E., Robinson-García, N., & Cabezas-Clavijo, A. (2011). Productividad e impacto de los investigadores españoles: umbrales de referencia por áreas científicas. Revista Española de Documentación Científica, 34(4), 505–526.
Jöreskog, K. G. (1973). Analysis of covariance structures. In P. R. Krishnaiah (Ed.), Multivariate Analysis (Vol. III). New York: Academic Press.
Kline, R. B. (2011). Principles and practice of structural equation modelling (3a ed.). New York: The Guilford Press.
Li, F., Yi, Y., Guo, X., & Qi, W. (2012). Performance evaluation of research universities in Mainland China, Hong Kong and Taiwan: Based on a two-dimensional approach. Scientometrics, 90, 531–542. doi:10.1007/s11192-011-0544-1.
Liu, N. C., & Cheng, Y. (2005). Academic ranking of world universities. methodologies and problems. Higher Education in Europe, 30(2), 127–136.
Lohmöller, J. B. (1989). Latent variable path modeling with partial least squares. Heidelberg: Physica-Verlag.
Luque-Martínez, T. (2013). La actividad investigadora de la universidad española en la primera década del siglo XXI: la importancia del tamaño de la universidad. Revista Española de Documentacion Cientifica, 36(4), 1–15. doi:10.3989/redc.2013.4.1046.
Luque-Martínez, T. (2015a). Actividad investigadora y contexto económico. El caso de las universidades públicas españolas/Research activity and economic context: The case of Spanish public universities. Revista Española de Documentacion Cientifica, 38(1). doi:10.3989/redc.2015.1.1135.
Luque-Martínez, T. (2015b). Horizon 2031. The University of Granada in Light of its V Century. Reflections on the Future of the University. Editorial Universidad de Granada. Granada (España). http://biotic.ugr.es/pages/horizon-2031.
Luque-Martínez, T., & Del Barrio-García, S. (2006). Plan estratégico de la Universidad de Granada. Editorial UGR: Granada.
Luque-Martínez, T., & Del Barrio-García, S. (2009). Modelling university image: The teaching staff viewpoint. Public Relations Review, 35, 325–327.
Luque-Martínez, T., & Doña-Toledo, L. (2013). What do graduates think? An analysis of intention to repeat the same studies and university. Journal of Marketing for Higher Education, 23(1), 62–89.
Marginson, S., van der Wende, M. (2007). Globalization and Higher Education. OCDE Education Working Papers No. 8, OCDE, Paris. doi:10.1787/173831738240.
Ringle, C. M., Wende, S., & Becker, J.-M. (2015). “SmartPLS 3.” Boenningstedt: SmartPLS GmbH, http://www.smartpls.com.
Robinson-García, N. (2014). Classifying and visualizing the disciplinary focus of universities. The invisible factor of university ranking. Doctoral thesis. Universidad de Granada.
Sanz-Casado, E. (2012). Lanzamiento del Observatorio IUNE, una herramienta para el seguimiento de la actividad científica de las universidades españolas. Revista Española de Documentación Científica, 35(3), 503–505.
Sanz-Casado, E., García-Zorita, C., Serrano-López, A. E., Efraín-García, P., & De Filippo, D. (2013). Rankings nacionales elaborados a partir de múltiples indicadores frente a los de índices sintéticos. Revista Española de Documentación Científica, 36(3), 1–18.
Sun, Y., & Liu, F. (2012). Measuring international trade-related technology spillover: a composite approach of network analysis and information theory. Scientometrics,. doi:10.1007/s11192-012-0860-0.
Torres-Salinas, D., & Jiménez-Contreras, E. (2012). Hacia las unidades de bibliometría en las universidades: modelo y funciones. Revista Española de Documentación Científica, 35(3), 469–480.
Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E. C. M., Tijssen, R. J. W., van Eck, N. J., et al. (2012). The Leiden ranking 2011-2012: Data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology, 63(12), 2419–2432.
Wold, H. (1982). System under indirect observation using PLS. In C. Fornell (Ed.), A second generation of multivariate analysis (pp. 325–347). New York: Praeger Publishers.
Zhu, J., Hassan, S., Mirza, H., & Xie, Q. (2014). Measuring recent research performance for Chinese universities using bibliometric methods. Scientometrics, 101, 429–443.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
See Table 4.
Rights and permissions
About this article
Cite this article
Luque-Martínez, T., del Barrio-García, S. Constructing a synthetic indicator of research activity. Scientometrics 108, 1049–1064 (2016). https://doi.org/10.1007/s11192-016-2037-8
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-016-2037-8