Applying data driven decision making to rank vocational and educational training programs with TOPSIS
Introduction
The 2008 financial crisis that hit the world's economies has had a particularly acute impact in Spain [1]. It is only since 2014 that Spain seemed to begin its recovery [2]. However, this recuperation is still far to be acceptable with regard to the labor landscape [3].
One of the main Spanish weaknesses that the crisis exposed was the so-called duality of the labor market. Thus, Spain is characterized by the existence of two very different types of workers. On one hand, long term workers on indefinite contracts, having both a very high job security and a very high cost for companies (especially in terms of dismissals) and usually with university studies even for jobs that do not require them. On the other hand, short term workers on temporary contracts or seasonal contracts with low wages and, in most cases, with very little training.
Another structural weakness of the Spanish economy unveiled during the years of the crisis was the fact that it had been relied heavily on two pillars: construction and tourism (and their associated services). This productive model had its main Achilles heel in the low level studies required in many of the jobs created in both sectors. Moreover, the relatively high wages that a worker could earn before the crisis, mainly in construction, led many young people to abandon their studies to work in these industries, without prior quality training. When the crisis arose and the destruction of employment reached unprecedented levels, Spain discovered that had to deal with a mass of unemployed, mostly young, people who, having no adequate training, had very difficult or even impossible reinstatement into the labor market.
This problem has been most pronounced in some regions of Spain as Extremadura. Extremadura is a European Union Objective 1 region located in western Spain that according to the Eurostat Regional Yearbook 2018,1 its GDP per inhabitant in relation to the EU-28 average is 61.47%, it has 23.7% of unemployment rate and, even worse, its early leavers from education and training of young people rate is 20.9% and its young people neither in employment nor in education or training rate is 20%.
In other countries where the economic model was more diversified, with large sectors of skilled employment and better trained workers, the crisis was less intense, the employment destruction less acute, and the recovery was faster. One of the differences between Spain and, particularly, Extremadura with respect to those countries is the importance they give to Vocational Eucation and Training (hereinafter VET). For the European Union, VET should “prepare young people for entering and successfully and sustainably participating in the labor markets as well as to enable high potentials (e.g. migrants, refugees, low-skilled and unemployed, inactive groups, including women) to stay and/or (re-)enter the labor market” [4]. In Germany, for example, VET studies are closely linked to the labor market, so that the majority of VET students are also trained in companies where, in many cases, they end up working. This means that there is a quarry of workers with a specific qualification for the needs of the labor market.
However, in the collective imagination of Spanish families, VET has been considered for decades a second-rate training and has not been much appreciated. This vision, together with the high drop-out rates, has caused a very marked duality in training: on the one hand people who either did not finish more than compulsory education or, at best, have VET studies (which in this last case are seen wrongly as a low level education), or people with university education who, due to the high unemployment rates suffered in Spain during the crisis (still persisting), are hired in positions for which such studies are not really required, causing another of the many big problems of the Spanish labor market, which is the overqualification of workers [5].
In order to try to alleviate the mentioned problems, the Government of Extremadura, providing its historical data, asked for a scientific analysis about the real impact of VET studies on accessing to the labor market with a two-fold goal: increase the resources of those VET studies with higher employment rate and promote such studies among their unemployed citizens enhancing the image of the VET studies that really help to get a job.
To this end, the aim of this work is to determine the efficiency of VET studies and the evaluation of the performance of VET graduates from Extremadura in the different degrees of VET programs in the labor market. Thus, we illustrate a data-driven multi-criteria decision-making methodology with the aim of classifying the different degrees of VET programs according to some criteria related to labor insertion. Concretely, we have applied TOPSIS (Technique for Order Performance by Similarity to Ideal Solution) [6] to this problem. TOPSIS is a well-known classical MCDM method, widely used by researchers and practitioners, that supports decision makers in performing analysis, comparisons and rankings to select the best alternative using a finite number of criteria. Moreover, since all criteria are weighted, their importance may be modified providing, thus, the flexibility for creating different rankings prioritizing different aspects. In our case, we have considered 8 different criteria whose values have been computed from a real dataset with more than 28,000 VET student records containing both educational and labor information. This is a key contribution for this work since the information obtained is enriched by real data instead of being based on questionnaires as similar TOPSIS approaches [7] or the REFLEX project in the field of higher education [8]. Furthermore, to the best of our knowledge, the relationship between VET programs and labor market has not been explored by previous works with such level of detail.
The rest of the article is organized as follows. Section 2 reviews some works related with predicting different outcomes using academic data, and some other works using TOPSIS as Multi-Criteria Decision Analysis method together with a weight assignment analysis, on different scientific domains. Section 3 describes the datasets used and the process applied to them for computing the different data used in our study. Section 4 describes the methodology followed to apply TOPSIS. Section 5 explains the influence of the criteria applied during the process. The results obtained and further considerations are detailed in Section 6. Finally, Section 7 concludes the paper.
Section snippets
Literature review
The analysis of educational data is considered today as one of the foundations to implement new educational policies and may be even more so in the future [9].
In this sense, there are several works focused on analyzing success or failure in the pre-university stages. Thus, one example of this topic is the work of Sen et al. [10] that tries to predict the outcome of Turkish high school students in the examinations of national selection that these students must perform. Another example is the
Data management
This section presents the datasets used to establish the different rankings presented in this paper together with the whole process followed to obtain them. To this purpose, a collaboration agreement was established with the data owners, the Education and Employment Board of the Government of Extremadura, to acquire the data. Based on this agreement, we established several meetings with them in order to identify the information available and needed to perform the analysis. Then, the process for
TOPSIS
The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is a non-parametric multi-criteria ranking method which is model-free and data-driven. Following the notation of [27], let us assume we have m alternatives A1, …, Am and n criteria C1, …, Cn, with weights ω1, …, ωn. Each criterion may be either a benefit (i.e. “the more the better”) or a cost (i.e. “the less the better”). Let J+ and J− denote the sets of indices j corresponding to the benefit and cost criteria,
Criteria influence determination
As was mentioned earlier, one of the most delicate parts in the design of a ranking score is the weighting of each of the components that define it. The use of expert opinion is common when determining those weights, however, the heterocedasticity and the correlation between the different criteria make these assigned weights rarely coincide with their real influence in the final score ranking [29].
In order to assess this influence, we are going to use two different approaches. The first one is
Data preparation
The original data set consisted on a total of 28.272 people graduating in 121 different VET programs from 1996 to 2016. Those VET programs were classified according to the professional family they are included in. For each VET program and for each graduating year the scores of each person verifying the conditions of the different criteria were obtained. Then the median of such scores was calculated and considered as the score of that program for that year in the corresponding criterion.
Conclusions
In this work we have proven the effectiveness of the TOPSIS method as a multi-criteria analysis tool for the classification of VET programs in Extremadura during the period 2009–2016. It is very important to emphasize the fact that having quantitative data available relating different databases from the Education and Employment Board of Extremadura, allows us to have more criteria on which to base the performance of the different VET programs. It should be borne in mind that in this type of
Acknowledgements
This work has been developed with the support of (i) Ministerio de Ciencia e Innovación (MCI), Agencia Estatal de Investigación (AEI) and European Regional Development Fund (ERDF): RTI2018-098652-B-I00 and RTI2018-093608-B-C33 projects, and (ii) European Regional Development Fund (ERDF) and Junta de Extremadura: IB16055, IB18034 and GR18112 projects.
J. M. Conejero received his PhD in Computer Science from Universidad de Extremadura in 2010. He is an Assistant Professor at Universidad de Extremadura. He is the author of more than 50 papers of journals and conference proceedings and has also participated in different journals and conferences as member of the program committee. His research areas include Web Engineering, Big Data or Ambient Intelligence.
References (33)
- et al.
Why are labor markets in Spain and Germany so different?
Econ. Model.
(2018) - et al.
Clustering and ranking university majors using data mining and AHP algorithms: a case study in Iran
Expert Syst. Appl.
(2011) - et al.
Predicting and analyzing secondary education placement-test scores: a data mining approach
Expert Syst. Appl.
(2012) - et al.
Data mining models for student careers
Expert Syst. Appl.
(2015) - et al.
Analyzing undergraduate students’ performance using educational data mining
Comput. Educ.
(2017) An extended topsis for determining weights of decision makers with interval numbers
Knowl.-Based Syst.
(2011)Extension of topsis to determine weight of decision maker for group decision making problems with uncertain information
Expert Syst. Appl.
(2012)Comparison of weights in topsis models
Math. Comput. Model.
(2004)- et al.
A fuzzy-hybrid analytic model to assess investors’ perceptions for industry selection
Decis. Support. Syst.
(2016) - et al.
Variance based sensitivity analysis of model output. design and estimator for the total sensitivity index
Comput. Phys. Commun.
(2010)
State dependent parameter metamodelling and sensitivity analysis
Comput. Phys. Commun.
A mathematical programming approach to fuzzy efficiency ranking
Int. J. Prod. Econ.
Income, unemployment, higher education and wellbeing in times of economic crisis: evidence from Granada (Spain)
Soc. Indic. Res.
Spanish public finances through the financial crisis
Fisc. Stud.
Opinion on the Future of Vocational Education and Training Post 2020
Measuring occupational mismatch: Overeducation and overskill in europe—evidence from piaac
Soc. Indic. Res.
Cited by (17)
ABAC: Alternative by alternative comparison based multi-criteria decision making method
2022, Expert Systems with ApplicationsCitation Excerpt :Few applications of MCDM in business are mining and mineral processing (Sitorus, Cilliers, & Brito-Parada, 2019), identification of weak banks (Tsagkarakis, Doumpos, & Pasiouras, 2021), supplier selection (Abdel-Basset, Manogaran, Mohamed, & Chilamkurti, 2018), material selection (Emovon & Oghenenyerovwho, 2020) and financial decision making (Mousavi & Lin, 2020). In management, few applications are employee candidates recruitment (Karam et al., 2020), ranking of vocational and educational training programs (Conejero, Preciado, Prieto, Bas, & Bolós, 2021), and career preference of students (Pekkaya, 2015). Several popular MCDM methods such as the Analytic Hierarchy Process (AHP) (Wind & Saaty, 1980), the Technique for the Order of Prioritization by Similarity to Ideal Solution (TOPSIS) (C.-L. Hwang & Yoon, 1981), the Preference Ranking Organization METHod for Enrichment of Evaluations (PROMETHEE) (Brans, Vincke, & Mareschal, 1986), the VIKOR (Duckstein & Opricovic, 1980; Yu, 1973), the Borda-Kendall (BK) (Cook & Seiford, 1982), the ELimination Et Choice Translating REality (ELECTRE) (Roy, 1991), the Weighted Sum Model (WSM) (Peter C. Fishburn, 1967), the Weighted Product Model (WPM) (Honarmande Azimi, Taghizadeh, Farahmand, & Pourmahmoud, 2014; Rao & Venkatasubbaiah, 2016), and the Weighted Aggregated Sum Product Assessment (WASPAS) (Zavadskas, Turskis, Antucheviciene, & Zakarevicius, 2012) are proposed so far.
Measuring project resilience – Learning from the past to enhance decision making in the face of disruption
2022, Decision Support SystemsCitation Excerpt :This research proposes a novel quantitative measure of resilience that serves as a decision support system (DSS) to assist decision makers in developing a predicative model that establishes benchmarks for evaluation, and forecasts the disruption and recovery profiles of projects based on the behavior of past similar disrupted projects. This paper thus responds to recent calls for research on data-driven DSSs [7,11] by developing a decision-making model that supports analytical decision making across a project's life. Indeed, we contribute to rational decision-making theory by constructing a model that integrates learning into the disruption and recovery profiles of projects in the face of disruptive events.
Performance evaluation of Turkish Universities by an integrated Bayesian BWM-TOPSIS model
2022, Socio-Economic Planning SciencesCitation Excerpt :Supplier selection [26–30], supply chain network design [31], blockchain service provider selection [32], waste management [33], maintenance strategy selection [34], risk assessment [35,36] are the most tackled problems by researchers using BWM and TOPSIS extensions. There is very limited interest in education [37,38] employed by either BWM or TOPSIS. Although scholars have widely paid attention to the BWM-TOPSIS integration, the BBWM side is still remarkably untouched, waiting for researchers to implement [39].
From insights to impact: leveraging data analytics for data-driven decision-making and productivity in banking sector
2023, Humanities and Social Sciences CommunicationsMulti Criteria Decision Making for Ranking Digital Vocational Education
2023, 4th Research, Invention, and Innovation Congress: Innovative Electricals and Electronics: Innovation for Better Life, RI2C 2023The Empirical Nexus between Data-Driven Decision-Making and Productivity: Evidence from Pakistan’s Banking Sector
2023, Cogent Business and Management
J. M. Conejero received his PhD in Computer Science from Universidad de Extremadura in 2010. He is an Assistant Professor at Universidad de Extremadura. He is the author of more than 50 papers of journals and conference proceedings and has also participated in different journals and conferences as member of the program committee. His research areas include Web Engineering, Big Data or Ambient Intelligence.
J. C. Preciado received the Ph.D. degree from University of Extremadura, Cáceres, Spain, in 2008.He works as a Researcher and Professor with the University of Extremadura, Cáceres, Spain, in which he is currently Vicerrector in charge of quality issues. He has authored more than 50 relevant publications in journals and conferences related to these areas. His research interests include model-driven development, Web engineering, Predictive Analytics, Smart Cities, and Business Intelligence.
A.E. Prieto is an assistant professor of Computer Languages and Systems at the University of Extremadura (Spain). He is a member of the Quercus Software Engineering Group. He received his BSc in Computer Science from the University of Extremadura in 2000 and a PhD in Computer Science in 2013. His research interests include Linked Open Data, Predictive Analytics and Business Intelligence. He is currently involved in various R&D&I projects.
M. C. Bas is working as a teacher in the Business Mathematics Department at the Universitat de València. She obtained her Mathematical Science degree and her Ph.D. in Statistics and Optimization. Her main researching areas are the multivariate analysis, the design and analysis of surveys, the sample design, the generation of data bases and the application of sensitivity and uncertainty analysis to the construction of composite indicators.
V. J. Bolós studied Mathematical Science at the University of Valencia, where he obtained a Ph.D. in Theoretical Physics. He has developed his career as a professor at the Universities of Extremadura, Granada and Valencia, where he currently teaches and carries out his research in the Business Mathematics Department. At present, his main researching areas are dynamical systems and the analysis of time series by means of wavelet tools.