Keywords

1 Introduction

Safety of employees in industries and organizations has become one of the primary focuses today. Employers aim at providing a safe work environment to their employees. To maintain that, employees must attend several orientations and trainings. The employers are very keen in developing and implementing safe work procedures and rules. However, safety can be enforced with the right person for the right job. For example, in a car manufacturing company, there are specific requirements and rules on how to assemble parts. The right candidate with the functional capacity and stability will not only perform the task safely, but will increase the efficiency of production. Even though candidates are put through a lot of interviews and evaluation during the hiring process to test their capabilities, the assessment of the their functional capacity helps the employer recruit the better suited candidates [14]. Our framework addresses how the functional capacity and performance evaluation can be achieved. The system uses computer vision techniques and Towers of Hanoi (TOH) for the evaluation purposes. Towers of Hanoi is a well-known executive function task that it is used to assess cognitive skills and “measure working memory and inhibition processes” [16].

Most industrial accidents are due to the improper training [12, 15]. A good training method is one important key to maintain safety. Colligan et al. [3] states that proper training contributes to the safety of employees. At the same time employers spend on an average $3,000,000 approximately every year to train workers [8]. It always has to be a win-win situation where it benefits the employers by investing a low budget for training and the quality of training maintained. Our framework was used for a comparative study between three training methods: (a) traditional (with a human trainer), (b) gamification (game-based training simulation), and (c) computer-aided training. The comparison was made to find a method that helps train users better. From the developed framework and analysis, data was collected to identify the advantages and disadvantages of each training method.

2 Related Work

Functional capacity evaluation of physically challenged individuals and veterans has existed to assess their stability. This has always been clinic-based and practiced only by therapists [6]. It is suggested that using functional capacity evaluation in the workplace, in addition to the traditional interview based evaluation, would provide better evaluation of the employees. Extensive amount of research explains how workers can be efficiently trained for an industrial job. Human-based training is popular and widely used today. This method produces results but at a cost [11]. Researchers stress on the importance and the usefulness of the Game-Based Training (GBT) and its impacts on employees [7]. GBT offers safe and effective way to improve cognitive skills [2, 4, 5]. However, it is less effective with jobs where hands-on experience and training is required, e.g. unpacking items and loading them on the shelves in retail stores. In this case, trainees would gain less experience in a GBT when compared to hands on experience in the store. The computer-aided training that we have developed is a combination of human and game-based training, and have the potential to overcome the aforementioned drawbacks. The developed system can give personal assistance to the trainee, and at the same time provide the trainee with hands-on experience.

Assessment of functional capacity is a problem that has gained attraction from many researchers. Similarly, the advantages and impacts of GBT methods are extensively evaluated [9]. For example, LEGO assembly task was used to assess employees in an industrial simulation setup [10]. Retail corporations, such as Walmart, have been seeking alternative ways to train low-skilled workers [17]. This includes interactive games and applications that employees use on their commute and at home, offering on-the-job coaching for employers. In case of NASA, the shortlisted astronaut candidates start their training process by taking computer-based training on using various vehicle systems prior to operating live systems to help the candidates recognize malfunctions and perform corrective actions [13].

3 Experimental Setup

The study consists of a webcam, a computer, a physical Towers of Hanoi (TOH) as shown in Fig. 1. A computer-game replicating the physical experimental setup was developed using Unity game engine. To make sure the task is not too challenging, the number of disks in TOH is set to 5. The minimum number of steps taken to solve them is 2n − 1, where ‘n’ represents the number of disks. This would allow the users to solve the TOH task in 31 steps. The Towers of Hanoi were in a stationary position facing the webcam. The towers were labeled from left to right as Column 1, Column 2 and Column 3 as shown in Fig. 1. An overview of the experiment setup and design is explained in a YouTube videoFootnote 1.

Fig. 1.
figure 1

Experimental setup

Full size image

4 Methodology

In this study, we compare the impact of three types of training methods on the planning and problem-solving abilities of a trainee while he/she performs the Towers of Hanoi (TOH) task. For this, the study consisted of two phases, the training phase and the testing phase. In the training phase, the participants were divided into three groups and were trained to solve the Towers of Hanoi with one of the three methods, that is, (a) traditional (with a human trainer), (b) gamification (game-based training simulation) and (c) computer-aided training. After training, the participants were given a break and then the test began. In the training sessions, the participants were guided to solve the task using the optimal number of steps (which is 31 steps). However, in the testing session, there were no restrictions on the number of steps. From the results of the training and testing phases, analysis was made as discussed in Sect. 5. The three factors considered for the analysis were; total number of moves (steps), total time to solve, and the number of errors made while completing the task.

4.1 Participants

The participants were undergraduate students in the Department of Computer Science and Engineering at the University of Texas at Arlington. There were no restrictions based on their age and gender. A total of 18 participants were divided randomly for each of the three training methods and another 10 participants were part of the prototyping phase of our framework. All participants had no-prior knowledge of TOH, and they completed both training and testing phase.

4.2 Towers of Hanoi Rules

By default, there are some standard rules to solve the Towers of Hanoi. No additional rules were added to make the game easier for participants. These include:

  • Move only one disk at a time.

  • A larger disk may not be placed on top of a smaller disk.

  • All disks, except the one being moved, must be on a tower.

  • User will use only one hand to deal with the disk.

4.3 Training Phase

The first phase was the training phase. The participants were provided with the rules of the TOH. Each participant was presented with one of the training methods which were randomly assigned prior to the study. During the prototyping phase, the 10 participants provided data and feedback which suggested that the participants struggled to understand and recall the technique to solve TOH. So, the framework was updated such that each of the remaining 18 participants were trained twice.

4.3.1 Traditional Training Method (Human Trainer)

With this method, the participants were trained with a personal human trainer as shown in Fig. 2. The trainer went through the steps verbally to solve the TOH with the participants. This training was timed and the number of errors while solving was recorded.

Fig. 2.
figure 2

Traditional training. Human trainer (left) gives verbal instructions to the participant (right) to solve the TOH task.

Full size image

4.3.2 Gamification (Game-Based Training)

This method of training incorporated a computer game. The computer game was a replica of the physical version of the TOH, as shown in Fig. 3. GBT has become very popular in the recent decade. Researchers have found that this method is highly effective in training and has produced great results [7]. The participants solved the TOH by using the computer mouse to click on the disks that they wanted to move, and click on the tower where they wanted the disk to be placed. The instructions of the game, e.g. ‘move the red disk to tower 3,’ were flashed on the screen for the user to follow. The game restricted the participants to make wrong moves, e.g. moving a disk to tower 2 instead of tower 3, and false attempts were counted. To notify the participants of wrong moves, the system displayed error messages on the screen and made sound notifications.

Fig. 3.
figure 3

Game-based training. Participant plays the unity game

Full size image

4.3.3 Computer-Aided Training

In this method, the participants were trained with computer-aided instructions. That is, instead of an individual trainer, participants were asked to solve TOH with instructions flashing on the screen. Every step performed was captured through a webcam to evaluate the accuracy of the steps from the instructions that appeared.

The system was implemented using MATLAB. A webcam placed in front of TOH capturing 10 frames per second recognized the disks and their positions. To identify individual colors, HSV (Hue, Saturation, and Value) color space was used. The size of the disks was considered to avoid shadow based errors, noise and other objects that impeded the frame. Next, to identify the position of each disk, the centroid of each disk was calculated and the tower to which it belonged was identified. The system considered the disks for evaluation only when they were in a stationary position. Each step was considered a separate state. Every time a change is made, the system compared the current position of the disks with the position of the expected state. If they matched, it was considered a successful move and the current and the past states were updated. If they did not match, it meant that the disks were not in the expected position and was considered an error. In such cases, the system asked the participants to go to the previous move or to the actual move, as shown in Fig. 4, and then the system proceeded.

Fig. 4.
figure 4

Computer based training. The system asks the participant to go to the previous move or to the actual move to be made when they fail to follow the instructions.

Full size image

4.4 Test Phase

Once the training was completed, all participants were asked to complete the task (without any assistance) using the physical blocks of TOH as shown in Fig. 5. The results of this phase helped us evaluate the functional capacity of the participants and the effectiveness of the training.

Fig. 5.
figure 5

Test phase. The system does not provide the participant with instructions.

Full size image

The detection of colors and the disks was implemented using the same algorithm as used in the computer-aided training. Here, the system kept track of the past and current state. Initially when the test began, there was no past state and the starting position was updated as the current state. For every move the system checked the position of the disks and compared it with the rules. If the position of the rings satisfies the rules, the system updates the current position as the new present state and updates the past state. With this feature, the system recognized the previous moves and in case of an error, the system prompted an error message to the participant with a sound notification as shown in Fig. 6.

Fig. 6.
figure 6

Test phase. The system warns the participant of illegal moves.

Full size image

5 Data Collection and Analysis

Data was collected from 18 participants. The data consists of the total number of moves (steps), total time taken to solve Towers of Hanoi (TOH), the time taken for each step, and the number of errors made by the participants. As mentioned in Sect. 4.4, all the participants received the same testing phase. Figure 7 shows the average number of steps each group of participants needed to complete the TOH. All the participants were trained to finish the task in 31 moves, and any extra moves were considered as errors. The average number of moves in the testing phase was nearly the same in all groups, and the participants performed extra moves in the testing phase compared to the training phase. Similarly, Fig. 8 shows that the participants took greater time to complete the testing phase compared to the training phase. It also shows that the participants took less time to complete the game-based task in the training phase. The reason may be explained by the fact that the participants did not interact with the physical TOH in the GBT, which resulted in less physical effort and time. However, the completion time in the testing phase was very similar in all the three groups. The results of the number of moves and completion times might indicate that the different training approaches did not have a major effect on how the trainees performed. It also indicates that performing the training twice had very low practice effect, since the participants took longer time and more moves to complete the testing phase compared to the training phase.

Fig. 7.
figure 7

Average number of moves (steps) each group of participants performed in both the training and testing phase.

Full size image
Fig. 8.
figure 8

Average time to finish the TOH task by each group of participants in both the training and testing phase.

Full size image

Figure 9 shows the average number of errors performed by each group. In the training phase, both extra and illegal moves (i.e. large disk placed on smaller disk) are considered errors, and in the testing phase only illegal moves are considered errors. The participants in the human training group performed with fewer errors when compared with the other groups. This may indicate, although the performance is very similar in all the three groups, the participants who were given the rules for the task by the human trainer could follow the rules better. In the GBT group, the game restricted the participants from making illegal moves. When the participants tried to make illegal moves, the disks stayed in the original towers/columns and did not move to the wrong columns. The lack of hands-on experience during GBT may have contributed to the increased number of errors in the testing phase.

Fig. 9.
figure 9

Average number of wrong/illegal moves (steps) each group of participants performed in both the training and testing phase.

Full size image

5.1 Performance of Trainees

The data collected during the testing phase can be used to create a measure of performance and functional capacity. This measure can be used to check if the trainee is a fit for the job. The employer can set specific thresholds for the number of errors allowed, completion time, and how the trainees perform (e.g. number of moves). In certain jobs, where high accuracy is required, the employer can have a low threshold for the errors and high threshold for the completion time. For instance, in an assembly line, the employer can set thresholds to tolerate 2% of errors and 5% of delay in production completion due to of human errors. Trainees who surpass these thresholds may require more training. In our experiment, participants with average performance are considered to have sufficient problem-solving skills. Table 1 shows the average values from all the participants, which are used as thresholds. When applying these thresholds, we found 47% of our participants surpassed these thresholds and they need more training before being able to perform the TOH on their own.

Table 1. Threshold values for the towers of Hanoi task.
Full size table

5.2 User Survey Results

At the end of every experiment, participants were asked to fill out a survey form about their experience with the system. From the survey, more than 90% of the participants liked the experiment as they were new to the TOH. They also stated that they required complete focus and concentration while solving the TOH with minimum steps and errors. When the participants were asked how they liked their training method, computer-aided training group had the highest rating, followed by the human trainer training group, and then the GBT group. Contrary to the above statement, when the participants were asked to rate how much their training method helped to complete the task, the highest ratings were received from the GBT group whereas the human trainer group had the lowest rating as shown in Fig. 10. Specifically, one of the trainees in the human trainer group commented that ‘listening to a human trainer is helpful, but it does not help me think on my own.’

Fig. 10.
figure 10

User survey result for the likeability and helpfulness of the training approaches.

Full size image

6 Discussion and Conclusion

The need to reduce the unemployment rate and accidents due to human factors have encouraged businesses and government agencies to invest in training employees [1, 18]. Recruiting and training the best employees with the appropriate skills would ensure individuals secure their jobs and make fewer mistakes. Assessment of employees’ performance may provide feedback to employees so that they can accomplish their job safely and efficiently. In workplaces, this may be applied to employee’s requiring increased problem-solving and planning skills to accomplish their job (i.e. inspection and maintenance jobs). The Towers of Hanoi task discussed in this paper may be used to measure general problem-solving skills. Similarly, having game-based and computer-aided training and testing are not applicable to all scenarios or too costly. Employers must consider to the skills evaluated and cost-benefit analysis to the company to determine its implementation.

This study compares the impact of three types of training methods on the planning and problem-solving abilities of a trainees while they perform the Towers of Hanoi (TOH) task. The authors observed that participants in the traditional training group showed higher rating in following the task rules. Additionally, using the computer-aided and game-based training provided relevant feedback. They provided the same performance results as the traditional training and with accurate record of performance. Using computer-aided and game-based training also provide the participants with ubiquitous training as well as reduced human supervision. Finally, the use of intelligent training approaches help support training individuals to make their jobs easier and improve performance.