Keywords

1 Introduction

1.1 Review

In only 10 years (2006–2016), the student population worldwide grew by almost 50%, passing from 146 million to 218 million students, representing a growth of 4.1% per year. The number of students on international mobility is rising steadily and students in the world are more and more numerous to study abroad. In 2016, almost 5.1 million of students were on the move around the world (out of 218 million) compared with only 2.9 million in 2006. The development of higher education in the world, the competition of establishments and their internationalization as well as the multiplication of agreements and equivalencies of degrees are as much factors of a rapid increase in degree seeking mobility. Thus, the massive local investment in education found in many countries contributes to the creation of regional education centres. They offer the possibility to a greater number of students from the same geographical area to access a higher education outside their country of origin and thus encourage student mobility. They set a quality higher education offer at proximity and at a lower cost. Those 5.1 million students who have crossed the borders to obtain a higher education degree, it is a symbol of the growing interconnection of higher educational systems and globalization and career opportunities. The main trends in global student mobility confirm the attractiveness of the English - speaking countries (United States, United Kingdom, Australia and Canada) and Europe remains the first destination for students coming from Asia (China and India in particular).

France hosts each year international students willing to pursue their higher education in different fields. Their presence compensates for the relative lack of talented French students. The state and the French educational organizations want to increase the number of students to improve the position of the French educational system vis-a-vis the Anglo-Saxon system, among others.

More than 300,000 students annually from 180 different countries got enrollment in the French Universities and Graduate Schools. These academic institutions receive thousands of applications of the students who have completed their high school, bachelor, or master’s degrees. This diversity of students’ profiles and their different academic background make accurate assessment an extremely challenging task. Up to now, the prior learning assessment of candidates and the processing of their applications are done manually and hence these tasks are time consuming and involve a lot of human potential errors in the selection. In addition, the distinctiveness of the French education system especially with its “Grandes Ecoles” that are unique, complicates the task of evaluation. International standard exams like SAT, GMAT, GRE are insufficient to conduct a full assessment.

In addition, being in the higher education system for the last 10 years and from personal experience I have noticed that the assessment exams that are organized by the hosting institution are not measuring the right indicators to detect the eligibility of international students to the desired higher studies programs. To me admitting relevant profiles is still an unresolved issue that should be taken into consideration. As per my knowledge and the state of the art, there is no automated intelligent system that performs a multi-criterion evaluation of international students by communicating to them. Knowing the proposed programs, computer trends and needs of students, this system should propose an adjustment of programs.

1.2 Need

What is missing? As per our knowledge existing systems lack of considering criteria such as cultural aspects, level of education in each country, motivation, real meaning of motivation letter, and emotional intelligence. In the light of the current state of the art and on our knowledge, there is no automated intelligent system that performs a multi-criterion evaluation of international students by multi-media and multi-modal interaction with them. Knowing the proposed programs, computer trends and needs of students, this system should also propose an adjustment of programs, according to the student’s level. Here comes the need for automated multi-criteria system for the evaluation of students’ knowledge and motivation (Fig. 1).

Fig. 1.
figure 1

International mobility growth

Full size image

Many researchers and universities have worked on several systems to evaluate the students’ profiles to provide coherent admission results and to process the maximum number of applications possible. One of these approaches is to compare the profiles of new applicants with those who have similar profiles and have already validated their academic programs. A second approach was based on ranking of applicants by using the available historical data and predictive analysis to detect the risk of admitting those candidates. Authors use these data to orient students toward a specified major or domain. In my opinion, all of these approaches do not serve to solve the main problem presented in this paper as they do not provide a subjective and adapted assessment exercises. Also, those systems do not evaluate international students’ attended universities and their learning outcomes. In addition, the decision support system implemented do not use the latest technology.

1.3 Students Mobility Risks

The potential drawbacks that globalization might have on education are increasing and becoming a main worry. As mentioned earlier, the number of international students is increasing, and students’ mobility is becoming a crucial phenomenon to obtain a good degree and secure a decent job. Universities have tripled their efforts to recruit and attract international students, but their educational ecosystem is still missing some fundamentals. Ensuring that all international students are admitted using an adapted admission system, receiving assistance and decent welcome from international faculty and staff, and building their career after are factors that reduce the negative impacts of globalization in education. Difficulties are summarized into 4 main categories. The first category is related to the fact that the current assessment is not adapted to the programs and hence it is measuring irrelevant indicators. The second category is that the evaluation criteria does not address the cultural aspects of candidates. The third is related to the limitation in the human capacity of treating an important number of candidates/applications over time. The last is how to reduce the impact of this mobility on education. Hence the major focus of the research is to find the automated system that addresses the 4 main difficulties already presented.

In this paper, we are going to present the related work, research methodology, challenge, and the proposed system for evaluating the students’ profile using knowledge management.

2 Challenge

To remove this lock and define an effective system architecture, a deep understanding of the admission principles and related contexts is mandatory. Existing experience and explanation of the admission problem from the educative point of view must also be considered. Difficulties are summarized into 4 main categories. The first category is related to the fact that the current assessment is not adapted to the programs and hence it is measuring irrelevant indicators. The second category is that the evaluation criteria does not address the cultural aspects of candidates. The third is related to the limitation in the human capacity of treating an important number of candidates/applications over time. The last is how to reduce the impact of this mobility on education. Hence the major focus of the research is to find the automated system that addresses the 4 main difficulties presented above.

Today applying to any degree seeking program in France or anywhere in the world nationally or abroad requires that candidates should undergo an admission procedure to assess the profile of candidates and to announce the final decision (admission/refusal of candidates). Admission systems vary from one country to another and from one institution to another. The characteristics of each country and institution shape the admission system, however there is a big percentage of commonality between these systems as they require the same traditional documentation, evaluation, information: admission and languages proficiency exams, interviews, CVs, transcripts, motivation letters, and recommendations letters.

2.1 Admission Systems

In the last 10 years, most academic institutions have automated their application process and gave the international and national students the possibility to apply online to a desired program. Applying to any degree seeking program in France or anywhere in the world nationally or abroad requires that candidates should undergo an admission procedure that starts from uploading the official documents requested by each institution till the announcement of results. Admission systems vary from one country to another and from one institution to another. The characteristics of each country and institution shape the admission system, however there is a big percentage of commonality between these systems as they require the same traditional documentation, evaluation, information: admission and languages proficiency exams, interviews, CVs, transcripts, motivation letters, and recommendations letters. The admission cycle can be summarized in the below figure:. Up to now, all institutions need the candidate information and exams results to decide of the admission status of candidates. The admission procedure that is widely used worldwide and in France is described as follows (Fig. 2):

Fig. 2.
figure 2

Current admission cycle

Full size image

  • Students apply online on the institution website by supplying all the relevant and requested documents.

  • Admission teams process the files following the order:

    • Relevance of the candidate to the requested major

    • Candidates high school or bachelor grades

    • Candidates experience and skills

    • Interview conduction (remote or face to face) to detect: genuineness, motivation, and capacity

    • Exams conduction to detect knowledge and practice

    • Financial status (mainly Anglo-Saxon institutions)

  • Results announcements.

2.2 Knowledge Blocks

As an entry point to understanding the admission system, we should look at 2 major knowledge blocks that contribute to the relevant evaluation of students’ profiles: the Curriculum Vitae (CV) and the online interview. The CV still is an important document that helps the admission committee identify important information about the candidates and provide knowledge on student’s academic and career path. The online interview helps the admission committee to detect motivation and validate the coherence and genuineness of the candidates’ vis-a-vis their CVs and profiles. For this purpose, the online interviews will be registered and rerun for the offline evaluation.

Curriculum Vitae (CV)

This block is very essential in the evaluation process. The amount of text that is generated every day is increasing dramatically. This tremendous volume of mostly unstructured text cannot be simply processed and perceived by computers. Therefore, efficient and effective techniques and algorithms are required to discover useful patterns. Text mining is the task of extracting meaningful information from text, which has gained significant attentions in recent years. With the rapid growth of Internet-based recruiting, there are a great number of personal resumes among recruiting systems. To gain more attention from the recruiters, most resumes are written in diverse formats, including varying font size, font colour, and table cells. However, the diversity of format is harmful to data mining, such as resume information extraction, automatic job matching, and candidates ranking. Supervised methods and rule-based methods have been proposed to extract facts from resumes, but they strongly rely on hierarchical structure information and large amounts of labelled data, which are hard to collect. Since there is no standardization in the structure of resumes, high precision in the Extraction of Information automatically from the resumes is very complicated. Resumes can be in different file types with any format without having restrictions in the domain. Many technologies have begun to fill the gap between human and computer language.

In the current systems the CV students are not taken into consideration as the evaluation logic only depends on the profiles of previous students who have succeeded a certain curriculum. Hence it depends on a comparative mechanism that might be valid for students coming from the same background but might fail for a diversified group of students. This block should provide the below information about each candidate:

  • Basic: to detect the country of origin of each candidate, age and gender. The country will be a crucial factor in the adapted evaluation since a cultural impact matters here.

  • Academic background: to detect the institution attended by the candidate and the highest degree obtained and the number of academic years after high school. This will help also to compile a list of academic institutions worldwide that will be ranked based on students’ success after enrollment and pursuing of classes.

  • Professional experience: to verify the experience and skills acquired during this experience and its relevancy to the degree obtained. This will help also to compile a list of companies worldwide that will be ranked based on students’ success after enrollment and pursuing of classes.

Beside the direct knowledge that will be extracted from CVs, text mining procedures will be applied to derive knowledge from the unstructured text by merging all the above listed information. The initial work on the CV could be divided into 3 phases. The fist phase is the CV extraction. The main role of this phase is to transform any CV document to a simple text data. The recommend phase is applying natural language processing with machine learning to label the text in the CV. The last phase will be applying rules to produce a simple table where each row represent a candidate and each column represent a CV subsection. Once this table is obtained, the next step will be the evaluation of CV in context of each application by using machine learning. We will apply statistics, analytic, semantic and natural language processing algorithm and output of this exercise could be a weighted mark that aggregate CV main parts.

Online Interview

This exercise will serve in providing several types of knowledge on candidates. As per our observation and knowledge the interviews exercises for admission are crucial and are still useful today for recruitment activities, in academic or corporate affairs. They are the most important element in the admission process that universities cannot avoid despite their costs. Interviews provide necessary knowledge on candidates and their advantages are they provide a strong tool to check the social capabilities of the student under an academic ecosystem. Interviews will good elaborated questions, can manage to get valuable information on the candidate, digging deeper in the skills of the subject and the capacity to use them efficiently in the academic and corporate environment.

The online interview will be used to evaluate the English language level of candidates, their motivation, their capability to present themselves and present a coherent project of life, and some easy behavior aspects. The following 3 groups of knowledge could be extracted and evaluated from pre-registered video interviews:

  • Detect the candidates’ oral ability by evaluating the first couple of minutes of their interview.

  • Analyze the candidates’ behavior in the video in terms of self-confidence and coherence.

  • Evaluate the candidates’ answers to the interview questions to detect their motivation and relevance.

The 3 knowledge groups would be based on motivation, skills, professionalism, feelings and integrity. Our proposition is that motivation can be detected from audio intensity and voice security. It can also be deduced from the diversity of answers on the different questions. For the skills group, this should be evaluated based on a pre-selected skills set that candidates mainly possess when they are applying to similar programs. Also, a cross verification exercises against the CV results will help validation the skills acquired by candidates. Professionalism can be detected from the camera stability, face positions and orientation angles. For the feelings, presents the list of feelings that has been detected during the interview. It can be detected in the audio according to the voice parameters and characteristics and the content can be analyzed to detect if the overall opinion and usage of words is “positive” or “negative”. To detect the candidate’s integrity and attempts to cheat, an estimation of the gaze angle will be calculated to verify if the candidate is reading from a paper or looking outside the screen. Cheating attempt can result in a very bad grade for the candidate. For the evaluation purposes, any video will be broken down into frames and analyze each frame on its own, saving the detected characteristics in a set of results. At the end of analyzing the video, we will get a huge set that contains all the result sets from all the frames. The idea is to retrieve the audio files from the registered interviews. The following techniques could be used to have a precise evaluation: speech recognition, Large Vocabulary Continuous Speech Recognizer, Speech analytic and Semantic Interpretation for Speech Recognition. The output of this simulator could be a weighted mark on a scale of 5 (poor, fair, good, very good, excellent for example). This mark could be an aggregate of the language, motivation, behavior, and character. There is a possibility also to convert to text to apply the text mining techniques.

3 Proposed System

3.1 Initial Architecture

The comprehension of the nature and contexts of the elements leading to the correct evaluation will guide the choice of knowledge models and processing methods. The proposed architecture will contain several communicating building blocks. Working on the algorithm requires treating a list of modules that build up the main architecture.

These modules can be in the below order:

  • Exploration of the elements gathered during collection phase and comprehension of the relations in system’s components. Collected elements include information found during the educational research part. Exploration should include knowledge discovery and required analysis, as well as behavioral detection. The architecture will be adapted based on the educational research results.

  • Generation of an adequate exam. The exam should consider the results found in the previous research and the academic profile of each candidate.

  • Generation of the GAAF (Global Admission Acceptance Factor). This is the specific measure of student knowledge and capacity, that may vary according to the requirements.

    Developing the system, the nature of data, information and handled knowledge must be taken into consideration. In the case of a rich database with a lot of decision factors, BIG DATA principles and procedures could be applied (data mining, predictive analytics, double agent, analysis of user behaviour). The system (or the algorithm) should help finding the auto-adapted evaluation criteria per candidate. The Machine Learning problem will be dealt to discover the methods to be used (Random Forests, Support Vectors Machines). The challenge would be to find or build an effective system for this type of evaluation, and to implement and test it on a couple of cases. The developed system will be validated based on real cases and on the integrated feedback experience. It will be parametrized according to each institution and according to the accumulated data received during processing of new applications. After the validation of the system, we are committed to design a modular demonstrator, reusable, scalable and generic to perform the tests and get the necessary feedback on this prototype. This system is developed based on the experience accumulated in EPITA International Master’s program and its student recruitment system (CRM) It is tested in this environment and could be extended for example to be part of another projects (Fig. 3).

    Fig. 3.
    figure 3

    System principles

    Full size image

3.2 Architecture

The above architecture was proposed at the beginning of the research. While starting the conception of simulators and building the model of the proposed system, this architecture has evolved and transformed into several formats. The first form was a simple decision tree that elaborate the main functions of the proposed system, as shown in Fig. 4.

Fig. 4.
figure 4

Decision tree

Full size image

To have a deeper vision of the decision tree, a new detailed architecture has been established that takes into account in a granular level all the aspects of the proposed system. This architecture represents a cross functional design that is composed of 8 phases and it involves all the knowledge blocks to be studied. The first phase is about receiving all the documents related to the candidate application. Those documents will be the main input of the evaluation system. They include the CV, transcripts, program choice, academic information, personal information, support letters and language records. The second phase and third phase are simply the information extraction to obtain the information model corresponding to each data type. In the fourth phase, machine learning algorithms will applied to generate the first evaluation engine for eligibility and CV assessment. The fifth phase is to simply present the result and for analyse them. In the sixth and seventh phase, deep learning algorithms will be introduced for the video behavioural analysis and video answers evaluation. The eighth and last phase will be the generation of the second evaluation system to produce the seventh results and generate the GAAF factor (Fig. 5).

Fig. 5.
figure 5

Detailed architecture

Full size image

4 Conclusion

In this paper, the integrated system of student’s evaluation using knowledge management approach is presented. The main aim is to assess how KM can help in a such system. By presenting and analyzing the 2 major knowledge blocks that constitute the evaluation system, we will be able to obtain an architecture that leads to an adapted evaluation of candidates. Also, this analysis will help us to validate the proposed solution using real cases and integrated feedback experience. The advantage of the proposed exam is that it does not assess only the aptitude and the knowledge of the candidate in a certain domain. It goes beyond the instantaneous evaluation of students to assess the experience, the skills acquired and the behavior in a multinational environment. The next step will be building up 2 simulators based on the CV text mining block and the interview evaluation block. These 2 simulators will be tested on hundreds of students applications we own to verify their outputs and validate the proposed algorithm. These 2 simulators will be crucial to building up the architecture of the proposed system and test in real case scenarios.