Keywords

1 Introduction

Technological advantages lead to tremendous changes in the production industry. Modern machine tools have been developing to exceedingly automated production systems with highly complex functionality due to the increasing demand for high production rates and quality. However, the human-machine interface is not adapted to these circumstances yet, resulting in a lack of intuitive controlling features, causing high levels of mental strain for the operator [1]. Mental strain can be summarized as the intensity of cognitive processing amongst all stages of human information processing during informational operations. According to the stress-strain concept it is defined as a working person’s subjective psychic reaction towards a given amount of stress. The quantity of stress hereby depends on several objective influence variables such as working and environment conditions. The reaction’s extend and the resulting user performance on the other hand are dependent on internal and external factors of the individual, and thus vary amongst different individuals [2]. In this context overstraining tasks lead to the same consequences for performance as subchallenging operations [3]. These circumstances point out the necessity for more intuitive human-machine interaction concepts regarding increasing complexity of production systems, respective machine tool controlling.

This can be realized by adjusting the human-machine interface. Technical principles for human-machine interaction design are given by ergonomic requirements, for instance formulated in different parts of the European standard “Ergonomics of human-system interaction”, representing usability guidelines for product designers. The standard is based on legal requirements of the European Commission regulated in the machine directive 2006/42/EC, which is to be implemented in law by every European State [4]. The standard DIN EN ISO 9241-210 (2011) describes the steps of the user-centered design process for interactive systems. According to this approach, usability just like accessibility of products, are dependent on the context of use, specifically on user requirements, task to be completed and environment requirements [5]. Since actual standards already deal with ergonomic information display, as well as information input, recommendations for interface design have already been implemented in several technical systems. Moreover, research about information perception has been comprehensively conducted within the last decades in numerous eye tracking studies prove. However, less effort has been spend regarding the implementation of central cognitive processes as a design dimension of HMI.

To include the user’s mental strain into HMI-design, a deeper knowledge of human information processing is required. In this regard three phases of information processing exist. The first phase is given by early processes including information perception. The perceived information is further processed by central cognition processes; before it becomes translated into a motoric reaction during the phase of late processes [6]. To support the machine operator within these processes, information is to be organized that perception, speed and ease of comprehension is optimized to guarantee that the operator perceives essential information and is able to intuitively translate decisions into actions [7]. Following this approach, information corresponding to human mental models can be processed easiest. A mental model is a mind construct, representing a reduced reflection of a part of reality in someone’s head. However, current navigation structures of machine tool interfaces are structured function-oriented, whereas human mental models tend to be task-oriented [8]. Therefore the function-oriented model must be transferred into a task-oriented model for every single operation. Designing an interface with a task-oriented navigation structure, would be more suitable to mental operating models, reducing mental strain. In addition, the software of current HMIs has been mapping the ever-increasing machine complexity, leading to depth and complex hierarchical navigation structures [9].

To study these effects, we investigated a preliminary laboratory study in prior research that showed effects of task-oriented HMI design on performance and errors [10]. Based on these findings, we implemented a field study to test results in real working conditions. Using the example of a cutting machine, the human-machine interface was analyzed. Regarding to ergonomic requirements we formulated design recommendations according to task-orientation. On this basis a software prototype for cutting machine interfaces was developed. This paper deals with a field and a laboratory control investigation, evaluating the prototype according to relevant usability objectives for software dialogues [11].

2 Method

An actual cutting machine controlling software was analyzed by usability experts and machine manufacturers as model case. In a first step the navigation hierarchy was visualized in a tree structure. The navigation tree contained up to 10 hierarchy levels arranged by functions. In the next step, the graphical user interface (GUI) was observed according to ergonomic principles. Moreover operators were interviewed concerning the actual interface. Finally, we summarized results to design recommendations for programming a task-oriented prototype [5]. For the evaluation, we conducted a field study in a company of a machine tool manufacturer with a cutting machine, as well as a laboratory study in order to identify results, caused by side effects of the field environment. Both software versions were tested with one of the most commonly operated workflow “Cut foam block manually” in industry applications. This standard workflow (see Fig. 1) consists of short, non-complex substeps to guarantee comparability of the setup and to maintain a more standardized evaluation.

Fig. 1.
figure 1

Standard workflow “Cut foam block manually”

Full size image

The main objective was to test the task-oriented HMI prototype compared to the actually used function-oriented HMI. Therefore usability was tested for both versions with the IsoMetricsS questionnaire [12], based on Standard DIN EN ISO 9241-110 [11]. We concentrated on a single workflow that did not imply any tasks, which neither demanded individual adaption of the GUI nor triggered error situations. Thus, we eliminated these dimensions from the questionnaire. Results for each objective were analyzed by mean and standard deviation and compared within the two HMI software versions. Furthermore we tested subjective mental strain with the Rating Scale of Mental Effort (RSME) [13]. The RSME is a discrete scale with values from zero “no strain at all” to 150 “unbearable strain”. Thereby lower levels of subjective mental strain implicate a better support of the human-machine interaction, and thus indicate a better usability. Participants were shown the RSME scale after each condition and we asked to answer their stress experience spontaneously.

2.1 Participants

The study was divided into two parts, a field study in a cutting machine company, and a laboratory study as reference group. For the field study 8 participants (aged between 26 and 65 years) with an average age of 43.25 years were tested. Seven of the tested participants were male, only one participant was female. The same number of persons participated in the laboratory study. In this study the average age of participants was 25.75 and the number of male and female participants was both four.

All participants in the field study were employees of the cutting machine company and therefore had good knowledge on cutting machines and software on average. Participants that were tested in the laboratory on the other hand predominantly had neither good knowledge in cutting machines nor in the implemented software.

2.2 Procedure and Task

One part of the study was conducted in a company of a cutting machine manufacturer. Both software versions were installed and tested on an HMI of a cutting machine. After a short introduction and a questionnaire for demographic questions, all participants completed the task with both software versions in alternating order. They were given a step-by-step instruction of the workflow, without any further details, to test the interface according to its usability. The other part of the study was conducted in a laboratory of the Institute of Industrial Engineering and Ergonomics RWTH Aachen University under the same conditions, as a controlling condition with a more standardized study environment. In this case we used the same software versions and participants had to complete the same workflow with both software versions on a computer representing the HMI. In this part of the study, we did not connect a machine tool to the interface. Thus, participants were given a verbal and picture-based introduction to the scenario. The task to fulfil was the workflow for cutting a foam block manually (see Fig. 1). The starting task was permuted between condition 1 and 2 for each participant.

In condition 1 we used the actual software version on the HMI (see Fig. 2). The graphical user interface screen is organized as follows: a wide work area that displays the cutting job in progress is arranged centered. Top left the position of all axises is displayed, as well as the machine’s lock status and work mode. Center left the job queues can be modified, loaded and started. At the bottom a navigation tool bar is located, which can be switched and changes as a whole, when different operations are needed.

Fig. 2.
figure 2

Screenshot of the actual software version for cutting machines

Full size image

In condition 2 the newly developed software prototype was installed (see Fig. 3). The HMI prototype was designed with task-oriented menu structuring and user centered design elements. In addition buttons were arranged according to their relevance regarding the workflow. As visible in the screenshot, similar to the actual GUI, a wide centered work area is located, visualizing the loaded cutting job. Information about axis values is rearranged top right. Top left position now implies relevant task-oriented operations for fulfilling the cutting job, which is to take into account this area as more attentive for western people that are used to left-to-right and top-to-bottom reading [8]. For the same reason, buttons for often needed or rather critical operations, such as “home positioning”, “start/stop process”, “lock/unlock machine”, “machine mode” or “start process” are located top center, unlike the actual HMI, where some operations could not be seen constantly, when navigating through the tool bar at the bottom. User specific options, for example language or user group are located next to these on top right. Instead of the operation tool bar at the bottom, the prototype has some general functions arranged on this position on the screen. Here the user can modify display parameters, such as zoom in and out to take into account individual visual user abilities, as well as general operations independent from the actual job load, such as “load/unload material”. Furthermore, the prototype GUI visualizes relevant machine errors through circuit symbols, I/O-coded, in red if an error is occurring on the right hand side, for an easier error handling (not of relevance for the actual study set up).

Fig. 3.
figure 3

Screenshot of the task-oriented software prototype

Full size image

3 Results

Figure 4 shows that execution time for completing the workflow in both studies was lower when using the prototype than using the actual software version. Furthermore, process times in the field study were longer in general, as expected due to machine operations in the field study.

Fig. 4.
figure 4

Time needed to complete the task in field and alboratory study

Full size image

For each participant the five relevant usability objectives described above (see Sect. 1) were evaluated by means of the IsoMetricsS questionnaire. Mean and standard deviation of the results are shown in Fig. 5. Results show that none of the objectives were assessed lower than 3 points independent from the software version used. Furthermore, (a) suitability of the task (from 3.2 to 3.4), as well as (d) conformity with user expectations (from 3.5 to 3.7) improved slightly by about 0.2 points in condition 2. Learnability (e) could be increased by about 0.7 points through the software prototype. Controllability (c) didn’t change in condition 2. Self-descriptiveness (b) decreased unnoticeable. However, the standard deviation decreased for all objectives.

Fig. 5.
figure 5

Mean and standard deviation of usability objectives in field study

Full size image

In the laboratory control group, results occurred to be more clearly (see Fig. 6). In this condition the minimum was about 1.1 points and thus lower than in the field study. Moreover the mean over all objectives had a wider range up to 3.94. The graph does not show a difference in suitability of the task (a), self-descriptiveness (b) or controllability (c) between the two conditions. For conformity with user expectations (d) and learnability (e) on the other hand, results show a clear improvement of 2.8 points for (d) and 1.7 points for (e). Similar to the field study, the standard deviation decreased for all objectives, except for learnability.

Fig. 6.
figure 6

Mean and standard deviation of usability objectives in laboratory study

Full size image

Results of the subjective cognitive load show only a slight tendency to lower values in both conditions (see Fig. 7). Though, the diagram shows a remarkable difference in the values for field and laboratory study. Participants of the field study experienced the task by about 35% lower in mental strain than the persons, who attended the laboratory study.

Fig. 7.
figure 7

Mean of RSME values when using actual software version and prototype in laboratory and field study

Full size image

Qualitative assessment of participants shows that more than half of the participants found it easier, more intuitive and clearly to work with the prototype version in both studies (see Fig. 8).

Fig. 8.
figure 8

Qualitative assessment of both software versions in field and laboratory study

Full size image

4 Discussion

Execution time decreased by using the prototype version compared to the actual software in the field study by 10%, as well as in the laboratory study by 20%. However, we could not prove significant time differences between the software versions.

Results of the field study show that both software versions meet usability objectives exclusively with values above the mean (3) on the five point scale applied for the questionnaire. However, findings do not explicitly show an improvement of usability objectives due to task-orientation of the human-machine interface for the prototype version in condition 2. Regarding the software prototype only “task suitability”, “conformity with user expectations” and “suitability for learning” show a tendency to higher values compared to the actual software version. The participant’s knowledge with respect to the machine and actual software, the small sample size and furthermore, influence factors in the machine hall, e.g. external distractions from other workers or commonly occurring noises could be a possible explanation for these findings, since results in the laboratory study were more definite. Values of standard deviation were lower in the prototype condition, than in the actual software version tested in condition 1 for all usability dimensions. Based on this, results for the prototype usability are more consistent over all participants. This allows a more reliable interpretation, also indicating potential for more efficient process scheduling in production planning by means of task-oriented machine tool HMIs.

Results for the laboratory study point out this fact: the dimensions “conformity with user expectations”, as well as “suitability for learning” show decisive differences by means of task-oriented human-machine interface design. For both dimensions participants rated the actual software version only with 1 of 5 points on average. According to correspondence with the tested usability objective the value is equivalent to disagreement. The prototype version on the other hand, was rated with higher values above mean regarding these objectives. Participants agreed to the prototype’s conformity with user expectations by valuing with 4 of 5 points.

However, findings for task suitability, self-descriptiveness and controllability are less conclusive. For these usability objectives no change can be observed. Regarding the purpose for the same task of both software versions, it seems appropriate, that quantitative results for task suitability are similar. Moreover, neither “self-descriptiveness”, nor “controllability” is affected by ergonomic improvements considered in the prototype software design, according to the participant’s impressions. The value was near average of 3 for both versions. Self-descriptiveness is an indication for intuitive software handling, which becomes especially obvious for novices, who are dependent on self-descriptiveness, if they are not further assisted. The fact that all laboratory participants were novices, who were indifferent about the quality of self-descriptiveness of the prototype with no difference to the actual software version, clearly shows the ergonomic potential that is not exploited yet. In addition, controllability principally requires system knowledge, as well as understanding of tasks, processes and consequences. Hence, lack of experience with machine tool interfaces, and thus a missing mental model of controlling operations can be regarded as reasonable to explain the findings. The assumption about a lack of expertise of persons attending the lab study becomes more probable according to this observed little deviation from the mean. As controllability was rated nearly equal for both software versions, participants seem to have experienced similar challenges interacting with, respectively controlling both systems. Inappropriate mental models definitely contain potential to positively influence the discussed effect, since it is the most dominant common factor within the group, though unproven yet. To concern the probable correlation, and to validate this hypothesis, future studies are required.

In contrast to the questionnaire results for usability objectives, we found a clearer qualitative evaluation in the interview results. Using the prototype version was perceived as easier and more intuitive, furthermore this version was perceived as clearlier arranged by more than half of all participants, even in the field study, where differences did not show up between both software versions.

Results of the subjective cognitive load do not differ between both software versions, what could be originated in the standardized workflow task with short term subtasks that do not lead to high strain levels independent of the implemented software. Obviously knowledge about cutting machines and software has an influence on subjective mental strain, because novices had a higher subjective cognitive load than experienced participants for both software versions.

The main limitation of the study is the small sample size of 8 resulting from the effort of conducting the study during business hours in the machine hall and therefore limited access to employees. To better compare results, we tested the same number of persons in the laboratory study, which leads to unclear finding, difficult to interpret.

5 Summary and Conclusion

This paper deals with a field study about usability of task-oriented cutting machine HMIs. Prior research has already shown the benefit of task-oriented human-machine interfaces for a more intuitive interaction by reducing complexity and supporting human mental models [1]. However, these works did not study the influence of task-orientation on usability objectives. A better usability is one factor that can improve strain level by providing less stress while interacting with the system itself. After our findings in the explorative preliminary study, this study can be seen as a pursuing study built upon our explorative preliminary study [11] to gain first important findings of the impact of task-oriented HMI design on usability objectives in real working conditions. In this study we tested effects among two groups of participants. Participants in the field study were employees, familiar with the current software, whereas participants in the lab study were novices.

The results of the IsoMetricsS revealed that task-oriented HMI did not have an effect for skilled workers, who are accustomed to the actual software, since none of the usability objectives showed an improvement for the prototype compared to the actual version. Longtime trained mental models for the actual software version could be seen as an explanation. The findings are different, however, when regarding the novices. In this case results show a usability improvement according to “conformity with user expectations” and “suitability for learning”, which proves that the prototype HMI had a more intuitive navigation structure, is easier to learn for unskilled workers and, therefore, reduces initial training and execution time. Regarding results for the standard deviation, however, results were spread more evenly for both groups for the prototype version, confirming the quality of the discussed results. Nevertheless, results mostly valued above mean by three points rating and still reveal potential for improvement for the prototype version. This should be considered for future work. The study design aimed at standardized conditions and time efficiency to save field resources. Thus, very short workflow subtasks did not lead to real stress situations for the participants. Further studies therefore should extend procedure time to evaluate effects on mental strain with clearer results.