1 Introduction

There are a variety of factors which can affect students’ mental health, such as violence and school bullying [2, 6], parental divorce [4], etc. Research has shown these factors, which attribute from the lack of care by teachers and parents, could cause children’s social anxiety problems, particularly for left-behind children in Chinese rural area [9, 10]. Emotion regulation is such a complicated procedure that involves decision making in each emotion generative process [3]. Untrained students may not handle their emotion regulation easily, which may result in the development of mental disorders. Previous research has shown that about 20% children have experienced mental disorders [1]. The typical mental disorders universally happened in children are a social anxiety disorder, selective mutism, obsessive-compulsive disorder and posttraumatic stress disorder [5]. Teachers or parents are suggested to have mental health training, since a number of previous research has shown positive results of mental health interventions at schools [2, 7, 8]. In the schools where there is a counseling center, the processes of tracing students’ mental health, taking Hall-Woodward Elementary School for example, are: (1) teaching lessons of guidance on counseling; (2) having individual or group counseling; (3) holding relevant school events; and (4) organizing guidance curriculum program [11]. Some schools also take questionnaires at the beginning of semesters to obtain students’ mental health condition. However, neither taking questionnaires nor providing courses could provide just-in-time intervention to help students manage their anxiety and stress which are common in schools worldwide. Once school bullying happens, we cannot expect questionnaire or psychology class can take effect immediately and pertinently. Some students incline to hide the fact of being bullied, which may trigger off a series of mental disorders like depression or anxiety.

Emotion regulation is a way to control and express one’s emotion in both internal and external way appropriately to meet his or her social demands [13]. People are suggested to meet socio-affective needs, cognitive needs and action needs [14], among which the appropriate expression of emotion is a part of them. Self-statement is beneficial for emotion regulation, and it provides a space for students to estimate themselves.

Motivated by the aforementioned studies, we design a system that prompts students to reflect their emotion in time, and provides teachers and parents with just-in-time responding to and intervening the emotional disorders in an effective time period. What we especially design for students is a Universal Windows Platform (UWP) application of emotion video diary. This diary applies Microsoft cognitive service so that it has emotion cognitive function. So the whole system is an assistive tool for teachers and parents to care about students psychological conditions.

2 Related Work

A video diary can save children’s time and help them to practice the ability of expression [15]. In some research, an electronic diary is used for analyzing emotions. Research such as [16, 17] predicts pain for children with Juvenile Idiopathic Arthritis by analyzing emotion regulation from an electronic diary. In our market survey, electronic diaries prompt users to accomplish questionnaires such as anxiety and depression scale [17], so that the applications can analyze the data from the questionnaires. Some other research read the facial emotion of children with autism and social phobia [18, 19]. This research motivates our system design in that video diary can be combined with an emotion recognition module to detect students’ mental disorders in time.

3 Our System: Emotional Management System

3.1 Design Motivation and Goals

Our system mainly targets at two groups of people: students and teachers. The main design goals of our system are to help students acquire and practice emotion regulation skills, and give teachers an early warning of students’ potential mental disorders.

3.2 Overall System Architecture

Figure 1 shows the overall system architecture. Students access this system from the application, called Our Happiness Index. It is a UWP application, requiring Windows 10 tablet or laptop with a front camera. We assume it can run Microsoft Emotion Tracking API, but from our preliminary testing the real-time emotion tracking system causes a severe network latency. So we redesigned the application to capture students’ facial image each 10 s, and analyze the facial expression based on the still images. Data comes out as a formation of scores of each affective categories (e.g. anger, contempt, disgust, etc. [12]). And the results will be stored in a local database. Students could access the application using their own mobile devices; in such case the local database only stores individual data containing personal information, video recording and emotion scores to ensure personal data protection. Only a few personal information which attempts to distinguish students and the emotion scores will be uploaded and synchronized on Azure Cloud SQL server. Scored emotion is visualized on a website (http://trickuth.azurewebsites.net/, last accessed 2018/01/27), where teachers can log into it to check students’ periodical emotion states. At this moment, the daily emotion states of a student are their highest-frequency states in a day. In fact, teachers may see the complete states in a given day and compute the average states, minimum (negative) states, etc.

Fig. 1.
figure 1

The overall system architecture of emotion management system

Full size image

3.3 Main System Feature on Emotion Computation via Face Tracking and Personal Video Diary

Figure 2 shows the general use of our system. The part of the system, the application named Our Happiness Index (Fig. 3), is designed for students. It is an emotion-oriented video diary application. We posit that when using the application, students not only make verbal expression but also facial emotion at the time when they are recording the video. Since the current application only uses face API, it provides a channel that allows students to express both positive and negative emotion. By expressing feelings, the application allows the students to observe their own emotion changes and raise the alarm when negative emotion continuously persists. The latter provides teachers windows of opportunity to work with the student to regulate their emotion accordingly.

Fig. 2.
figure 2

System sketch maps with Microsoft Azure services

Full size image
Fig. 3.
figure 3

Demonstration of Our Happiness Index for students

Full size image

The other visual part of the system is a web application for teachers. In the website, teachers can see all students’ emotion states at a time, and to check everyone’s emotion state for a particular day or a longer-term (Fig. 4). Thus, when some students fall into a depressive state, teachers can have enough preparation of mental health intervention.

Fig. 4.
figure 4

Visualized report of a student’s daily and long-term emotion

Full size image

4 Discussion

Our system is a combination of video diary as a form of emotion cognitive service and mental disorder detection. Comparing with a paper diary, the video diary is more convenient and flexible for students. It also promotes students to speak confidently facing a camera, so that they reflect themselves and get to know how to regulate and manage their emotion. At the same time, teachers benefit from the system by acknowledging students’ emotion state and prepare for psychotherapy intervention.

However, due to the operation mechanism of our system, the data we acquired from the emotion was not always accurate. Captured images cannot represent students’ dominated emotion in the recording. What’s more, emotion in the recording also cannot represent the emotion all day long. Hence it could only be an assistive system to help teacher judge students’ emotion states at a glance, but we do not ensure the results from the system are absolutely right. In other words, professional psychotherapy is still essential for curing mental disorders without a doubt.

Moreover, the chances are that emotion regulation is related to trait resilience [6]. Therefore, we need to give students proper time, let them recover by themselves, then offer help when necessary.

5 Concluding Remarks

We present an emotion management system for assistive use on mental health tracking. The major group is students in schools because they lack the ability to regulate emotion and could easily have mental disorders. Our system is capable of picking up persistent negative emotion which thus provide windows of golden opportunity for teachers and parents to intervene on time.