Unobtrusive assessment of stress of office workers via analysis of their motion trajectories

Abstract

Stress is the second most frequent work-related health problem in Europe, and it is important to assess stress risks relatively early to diminish negative consequences. Automatic detection of everyday stress is a challenging problem, however, as both stress perception and stress manifestation notably vary between individuals. Therefore stress detectors, trained on data of each person separately, usually achieve notably higher accuracies than non-personalised ones. Unfortunately, this accuracy gain requires collecting too large sets of labelled training data to realistically obtain from end users. In addition, the majority of current stress detectors exploit physiological or mobile phone data: the latter approach increases battery consumption compare with normal phone use, and the former requires to wear additional devices and to charge their batteries. Unlike previous work, this work proposes genuinely unobtrusive personalised stress detection system, based on use of environmental sensors and unsupervised training of hidden Markov models (HMM) classifier and hence requiring neither sensor maintenance nor data labelling efforts from end users. In the experiments with real life behavioural data of office workers, collected during 10 months, the proposed system achieved 67% accuracy of classifying each day as stressful vs. normal and 95% accuracy in classifying months, thanks to discovery of novel characteristics of motion trajectories, indicative of stress.

Introduction

Stress is the reason for at least half of lost working days in European enterprises, and it is important to detect stress relatively early [1]. Then certain preventive measures could be taken, for example, a stressed person may be offered a relaxation programme, or some of his/ her tasks may be delegated to less busy colleagues. Hence recently development of stress detection methods, suitable for real life settings, became an active research area. Two main research directions are analysis of physiological data (e.g., heart rate, skin conductance, respiration, pupil diameter etc.) and analysis of behavioural data (e.g., mobile phone usage, posture, facial expressions, computer keyboard/ mouse usage, motion, social communications etc.) [1]. To date, main sources of behavioural data in field studies were mobile phones and microphones, whereas use of cameras and other environmental sensors was mainly studied in the labs. Methods, developed for lab data, cannot be directly used in real life, though. Lab studies typically last a few hours and thus mainly aim at detecting short periods of intense mental work, whereas in real life stress can be caused, among other factors, by insufficient variety of work [2], which is closer to lack of intense mental work than to its excess. In addition, in the labs test subjects are always positioned conveniently for the cameras, but in real life sensors should be positioned conveniently for the users, which inevitably results in occlusions.

Methods to use physiological data for stress detection cannot be directly transferred from the labs to real life settings either. Various daily life activities (eating, drinking, caffeine intake, conversation, motion etc.) influence physiological parameters [3]; physical motion causes also device shifts, notably deteriorating data quality. For example, in a recent study each test subject was wearing two wrist devices during one month, but collected physiological data only allowed to classify test subjects into two groups: more and less stress-prone individuals [4]. One more real life challenge is the fact that stress perception is highly subjective [5], [6] and stress manifestation in physiological and behavioural data is also distinct to every individual [1], that’s why in many lab tests [7], [8], [9], [10], [11] and field studies [6], [12], [13], [14], [15] person-specific stress recognition models achieved significantly higher accuracy than general models (i.e., models, trained on data of many subjects and not adapted to each individual). In case of using behavioural data average differences between accuracies of personal and general models may exceed 20%, but to date, such accuracy gains were achieved by fully supervised training, requiring fairly large sets of labelled data (nearly 100 self-reports were obtained from each subject in [8], [12]). This is quite big effort considering that human behaviour evolves with time and thus stress detection models should be periodically updated to adapt to such changes. Test subjects rarely provide large numbers of self-reports even once: for example, Adams et al. [16] observed that test subjects on average respond to less than one third of system prompts. Thus realistic stress detectors should not require end users to provide self-reports in large quantities, but the majority of existing studies nevertheless employed fully supervised machine learning methods.

As it is hardly possible to train “one-fits-all” model of normal behaviour and “one-fits-all” model of stressed behaviour, behaviour-based stress detectors should be person-specific, but should be trained using little or no labelled data, in contrast to the current practice to employ fully supervised classifiers. To date, however, only a few works proposed methods to reduce the need in labelled data of each target user. The majority of them use labelled data of similar persons to obtain stress detector for the target user, but this approach does not work well if a target user is insufficiently similar to other persons or if chosen number of similar subjects is not optimal [11], [15]. Unsupervised stress detection was studied in [17]: stress was detected as temporal deviation from normal HRV (heart rate variability) data, and periods of high physical activity were excluded on the basis of accelerometer data. Vildjiounaite et al. [18] proposed unsupervised method to detect stress as anomaly, but only high intensity stress was recognised this way. As long-lasting stress of low intensity may have equal or greater impact on health as a short-term high stress [19], recognition of just high stress does not suffice for long-term wellbeing monitoring.

The main contribution of this work is the following: first, it suggests a realistic setup of depth cameras for real offices, where workers do not sit all the time near a single computer and where furniture obstructs view on a whole body. This setup is also fairly privacy-safe because depth cameras do not point at faces of the test subjects; instead, depth cameras are positioned in the offices under the ceiling so that they have top-view side-view of the monitored subjects and detect nothing but head trajectories. Second, this work compares various characteristics of motion trajectories and suggests novel features, indicative of stress. Then this work confirms the feasibility of the proposed unsupervised method to learn person-specific stress recognition models by presenting results of ten-month-long monitoring of three different offices: a single-occupancy one and two double-occupancy offices. In one of these double-occupancy offices only one person was monitored, whereas in another office both occupants were monitored by a single depth camera. Experimental results demonstrate the ability of the proposed approach to detect stress of individuals with different work duties (senior researchers and organisational managers) and different stress perceptions (percentage of stressful days ranged from 23% to 51% for different subjects despite they all had challenging tasks). To the best of our knowledge, this is the first long-term real life study into unsupervised stress recognition using motion trajectories, acquired from depth cameras.