Keywords

1 Introduction

With the rapid development of Internet technology and the implementation of “Internet +” actions, a variety of web applications are widely used. However, there are also many information security issues which cause huge economic losses. In recent years, identity authentication based on user behavior has become a hot topic in the field of network user authentication research. The approach only needs to use the human-computer interaction device to collect the data of the end user behavior, and then analyze the network user behavior characteristics to perform the user identification.

Most researchers have analyzed user behavior from different perspectives for user authentication [1, 2]. For example, researchers analyze the user’s mouse dynamics to achieve static or continuous authentication of user identities. Although the authentication based on user’s mouse behavior characteristics has a good accuracy rate, the emotional state of the user’s human-computer interaction is not considered.

In many scenarios, users will perform human-computer interaction under different emotions even for the same task [3, 4]. It becomes more difficult to authenticate the network user by behavior data under different emotions, because it is not sure whether the network user’ behavior pattern will be affected by emotions, and also the extent to which different emotions change the network user’ behavior. Therefore, this study analyzes the user’s mouse behavior characteristics to measure the identity trust of users under different emotions, and to verify whether there is a significant difference.

2 Literature Review

Authentication based on user’s biological characteristics is another guarantee of network information security. Researchers analyze the data of users’ mouse and keyboard to build a biometric system [1, 5]. Therefore, most researchers require users to use the mouse to complete a specified task, then extract velocity vector features and detect mouse movements to identify the user [6, 7]. The results of the study show that although the same task is completed, the mouse movements of different users are different. Then, researchers extract more mouse behavior characteristics to build user models. Mondal and Bours [8] extracts five characteristics (type of action, direction, speed of the mouse action, reciprocal acceleration of the mouse action, traveled distance in bins) of mouse behavior, and uses algorithms to establish different levels of trust models to achieve continuous identity authentication. Feher [9] divides the mouse operation into three levels, extracts features according to motion state and mouse function, and then constructs a classifier for each action type to construct a multi-level model for continuous authentication. In the experimental control environment, static authentication can be performed based on mouse behavior characteristics. However, Jorgensen and Yu [10] believed that the mouse operating environment should not be controlled, and the user’s natural behavior should be collected for continuous authentication. Whether or not in the experimental control environment, authentication based on mouse behavior is to analyze whether the deviation of mouse behavior is within the normal range. Although the authentication based on user’s mouse behavior characteristics has a good accuracy rate, the emotional state of the user’s human-computer interaction is not considered.

In addition, users have different emotions, which is very important in the process of human-computer interaction. Some researchers explore the changes of users’ emotions to design human-machine interface for higher user experience and satisfaction [11]. And scholars have conducted research on emotion recognition in human-computer interaction [12, 13]. However, at present, most of the research is to collect the digital content (text, audio, pictures, etc.) of user interaction and establish an emotional recognition model [14], so that emotional content can be divided into positive or negative, or objective (neutral) [15]. Although most researchers agree that behavior in human-computer interaction can distinguish the user’s emotions, it is difficult to analyze from mouse behavior. There are only a few studies based on mouse behavior to measure user emotions, and few mouse behavior features are selected in the exploration [16]. In the current study, it is more to analyze the mouse behavior from the time dimension to predict the change of emotion, instead of analyzing the motion characteristics of mouse movement. However, simple dimensions make it difficult to get accurate results in a complex website operating environment. To deal with this problem, considering the complex environment of using computers and selecting more mouse behavior characteristics to build a model, the relationship between emotion and mouse is more clearly reflected.

Therefore, this study considers the emotional changes in human-computer interaction, and explores the impact of different emotional states of users on the accuracy of authentication based on mouse behavior.

3 Methodology

3.1 Independent and Dependent Variables

The independent variable was emotional state in the experiment, and was designed in group. Emotion state had three levels: neutral emotion, positive emotion, negative emotion. In the experiment, participants need to complete tasks after their emotions were aroused, so the extend of emotional arousal was the focus of consideration. Therefore, video was used to arouse user’s emotional changes, and Facereader was used to detect participants’ emotional changes and duration. In order to achieve the purpose of human-computer interaction after the participants’ emotions were aroused.

The dependent variable was the accuracy of authentication based on the characteristics of the mouse sliding behavior. The accuracy of authentication was the result of classifying the characteristics of mouse sliding by random forest classifier.

3.2 Participants

In the early stage of the study, ten postgraduates majoring in laboratory-related subjects were invited to conduct multiple emotional tests and pre-experiments. Eighteen college students (9 males and 9 females) form Chongqing University were recruited to be participants during the formal experiment. Their average age was 22.67 (SD = 0.796). They have the ability to operate computer skillfully, and the average year of experience is 4.48 (SD = 1.167). Moreover, 90.47% of participants believed that video could stimulate human emotions, and 9.53% of participants expressed uncertainty.

3.3 Equipment

Preliminary experiments and formal experiments were conducted in the human factors engineering laboratory of Chongqing University. In the preliminary experiment, participants’ emotions were detected by installing facereader software on a laptop (HP ProBook 440 G4). Facereader is a software that automatically analyses facial expressions (Neutral, Happy, Sad, Angry, Surprised, Scared, Disgusted). Facereader can get the percentage value of the corresponding emotion based on the user’s facial expression. In order to create a real user operating environment (experimental environment) without changing participants’ usual mouse operating habits, participants used their own computers and mouse. Finally, interactive data were collected using experimental equipment (HP ProBook 440 G4).

3.4 The Design of Emotional Arousal

In this experiment, three videos were selected to stimulate users’ neutral, positive and negative emotions. Previously, 10 participants were invited to test and facereader was used to detect the effect of video on emotional arousal and to verify the duration of emotional arousal. Facereader analyzed participants’ facial expressions while watching videos to get percentages of seven basic emotions (Neutral, Happy, Sad, Angry, Surprised, Scare, Disgusted). Neutral was classified as neutral emotions, Happy and Surprised as positive emotions, and the rest as negative emotions.

After repeated tests and adjustments, video 1, video 2 and video 3 were finally used to stimulate users ‘neutral, positive and negative emotions. The effect of emotional arousal was showed using percentage of Facereader tests (as shown in Fig. 1). For the first 120S, the percentage of emotions was more than 50%, so the three videos were considered to be successful in stimulating participants’ emotions. From 130S to 330S, neutral emotions and negative emotions remained, while positive emotions showed a downward trend in 290S. Therefore, the emotions stimulated by the video can last for about 2.5 min. The mouse operation data in the 2.5 min was also collected in the formal experiment.

Fig. 1.
figure 1

The percentage and duration of video-inspired emotions

Full size image

3.5 Experimental Systems and Tasks

The experimental system was a self-built academic exchange website of the research group (http://www.cquieaml.com/). The front-end web page (as shown in Fig. 2) was developed using HTML and javescript and consists of seven parts (academic research, scientific research results, corporate communication, forum interaction, research team, resource sharing, management center). The structure of the website was well structured and the content of the website was rich. Users can browse papers, post or comment on the website. Therefore, this experiment can create a real user operating environment and get the most realistic user behavior data. Javascript codes recorded users mouse data, like click and move, and then transformed the data into back-end sever and saved them.

Fig. 2.
figure 2

Interface of experimental system

Full size image

Each participant needs to complete two tasks in the event that one emotion was activated, each task needs to be operated on the website using the mouse. For example, browsing the post content and posting, this is a simple task. Participants open the specified post, browse, and then copy a piece of text for evaluation and posting. The more difficult task was to test the participants’ familiarity with the site. Participants need to answer questions set in advance, and the answers to the questions require the user to find them in the seven sections of the website. In order to avoid the learning effect, the tasks in different emotions had the same form, but the content was different.

3.6 Procedures

The whole experiment process was as follows: First, the experimenter introduces the purpose, content and precautions of the experiment to the participants. Second, participants were required to fill out a basic information questionnaire, including basic personal information (age, gender, etc.), computer mouse operation, and so on. The experimenter introduced the experiment task to the participants and instructed the participants to complete all the tutorial operations. Participants were required to perform continuous task operations immediately after watching the video, and there was no mandatory sequence for completing two tasks.

After completing the tutorial, enter the formal experiment, the experimental sequence was as follows: It was conducted in four days (four times). On the first day, the participants completed the tutorial independently, familiar with the task flow and the structure of the website, and avoided the influence of the experiment on the mouse operation due to unfamiliar experiments. Then, in three days, watching the video to stimulate different emotions to complete the task to avoid the interference of the subject’s fatigue or emotional changes. After the video 1 was watched the next day, the task operation was performed. After video 2 was watched on the third day, the task operation was performed, and after the video 3 was watched on the fourth day, the task operation was performed. The whole experimental process creates a real user operating environment (experimental environment), in order not to change the usual mouse operation behavior habits of the test.

3.7 Data Processing

The original mouse data collected by participants interacting with the website includes five values: the type of mouse event, the x-coordinates of mouse pointer, the y-coordinates of mouse pointer (y), the time (t) of mouse event and user ID. Most researchers believe that the basic events of mouse behavior were click and slide. This paper focuses on move sequences to better illustrate user mouse behavior. Firstly, data of mouse behavior were cleaned, classified, and featured through R programing, and the mouse data with emotional tags (2.5 min) was sorted out. Then, according to the mouse action, the move sequences characteristics (as shown in Table 1) were calculated.

Table 1. Description of Mouse move sequences characteristics
Full size table

The participant’s move sequences characteristics values were calculated according to the formula, and then the average of each of the above feature values was taken to reflect the mouse behavior. The accuracy of authentication was based on the characteristics of mouse movement to establish model calculation results. Several machine learning methods were compared, and the random forest algorithm was chosen to build the model because of the high precision. Firstly, the whole mouse sliding operation characteristics value of eighteen participants on the first day was used as the training set, and the random forest algorithm was used to establish the model. Then the mouse sliding characteristics value (2.5 min) under different emotions was tested as the data of the test set. The accuracy of authentication based on the characteristics of the mouse sliding behavior under different emotions was obtained. Finally, the accuracy rate of eighteen participants under different emotions was conducted using repeated ANOVA.

4 Results and Discussion

4.1 Influence of Emotion on the Accuracy of Authentication

First, the confusion matrix of 18 participants in neutral emotions, positive emotions, and negative emotions was obtained by random forest classifier (as shown in Figs. 3a–c). The average accuracy of authentication under neutral, positive and negative emotions were 83.6%, 80.3% and 81.9%, respectively. At the same time, the accuracy of authentication based on mouse sliding behavior of 18 participants in three different emotions was obtained (as shown in Table 2). Then, the accuracy of 18 participants’ authentication under three emotions was analyzed by variance analysis. The results show that different emotional states of users have no significant impact on the accuracy of authentication. Then, the accuracy of authentication of the 18 participants in the three emotions was analyzed by ANOVA. The results show that there was no significant difference in the accuracy of user authentication under different emotional states (F(2, 34) = 0.551, p = 0.582 > 0.05).

Fig. 3.
figure 3

(a) Confusion matrix under neutral emotion. (b) Confusion matrix under positive emotion. (c) Confusion matrix under negative emotion.

Full size image
Table 2. Accuracy of authentication of 18 participants in three emotions
Full size table

4.2 Discussion

This study examined the use of the mouse in neutral, positive, and negative emotional states. Then, the trust of the user authentication is measured based on the user’s mouse sliding behavior under different emotional states.

It can be seen from Table 2 that the accuracy of user authentication based on the mouse sliding characteristics under different emotions is different. Most participants have a high accuracy of authentication, and only a few participants have low accuracy of authentication under different emotional states. It can be seen from the results that some participants have similar and higher accuracy of authentication in neutral and positive emotions, while their accuracy in negative emotions is lower. For example, participants 9 and 13. Participants No. 13 achieved 100% accuracy in authentication under neutral and positive emotions. It may be because the mouse operations of these participants are more affected by negative emotions. Some participants had a very high accuracy of authentication in negative emotions, but very low in positive emotions. For example, participants such as No. 1, No. 5 and No. 15 may be because their bodies are relaxed at the same time when they have a positive emotional state, thus affecting the mouse operation. Among them, the accuracy of authentication of participants 6 in the three emotions is basically the same and low, around 0.71. It may be that emotional changes have a greater impact on the mouse behavior of participant 6. As to the accuracy of authentication, the average accuracy of 18 participants was 83.6% in neutral emotional state, 80.3% in positive emotional state and 81.9% in negative emotional state. The accuracy of authentication fluctuates slightly under different emotions. Although the user’s authentication accuracy is different under different emotions, the credibility of authentication based on mouse behavior is not affected.

In general, the user identification model based on the mouse movement feature calculates that the accuracy of authentication is different under three different emotional states. And this difference is the content of experimental research. However, the user’s mouse behavior data with different emotions for human-computer interaction has no significant effect on the results of the identification research.

5 Limitations and Future Work

Three limitations of this study are noted. First, the study was to stimulate user emotions through video, but different participants may be induced with different emotional intensities, which may bias the results. Future studies may explore the effects of different intensity levels of different emotional states.

Second, this study recruited participants who were students from Chongqing University, regardless of age or occupation. Future studies may extend this study to other populations to check generalizability of findings from this study.

Third, this study only considers a few classification algorithms. Future studies should try to use different classifiers and prediction methods to improve accuracy and reliability.

6 Conclusions

Authentication based on mouse behavior is a guarantee for network information security. But the mouse behavior is affected by the user’s emotions. Hence, this study aims to explore the effect of user emotions on the accuracy of authentication based on user mouse behavior. In order to achieve this goal, this study conducted an experiment to explore the user’s mouse behavior under different emotions.

The experiment results show that there are differences in the accuracy of authentication based on the user’s mouse sliding behavior in three different emotional states, but the difference is not significant. Although the accuracy of the authentication of the 18 participants in neutral emotions, positive emotions, and negative emotions fluctuated greatly, the average accuracy rates were 80%, 81%, and 84%, respectively. The results also show that although the user performs human-computer interaction under different emotions, it will not essentially affect user authentication. Therefore, it can conclude that measuring network user trust via mouse behavior characteristics under different emotions is credible.