Skip to main content
SpringerLink
Log in

Toward the Analysis of Office Workers’ Mental Indicators Based on Wearable, Work Activity, and Weather Data

  • Conference paper
  • First Online:
Sensor- and Video-Based Activity and Behavior Computing

Part of the book series: Smart Innovation, Systems and Technologies ((SIST,volume 291))

Abstract

In recent years, many organizations have prioritized efforts to detect and treat mental health issues. In particular, office workers are affected by many stressors, and physical and mental exhaustion, which is also a social problem. To improve the psychological situation in the workplace, we need to clarify the cause. In this paper, we conducted a 14-day experiment to collect wristband sensor data as well as behavioral and psychological questionnaire data from about 100 office workers. We developed machine learning models to predict psychological indexes using the data. In addition, we analyzed the correlation between behavior (work content and work environment) and psychological state of office workers to reveal the relationship between their work content, work environment, and behavior. As a result, we showed that multiple psychological indicators of office workers can be predicted with more than 80% accuracy using wearable sensors, behavioral data, and weather data. Furthermore, we found that in the working environment, the time spent in “web conferencing”, “working at home (living room)”, and “break time (work time)’ had a significant effect on the psychological state of office workers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Li, B., Sano, A.: Extraction and interpretation of deep autoencoder-based temporal features from wearables for forecasting personalized mood, health, and stress. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4(2) (2020)

    Google Scholar 

  2. Swain, V.D., Saha, K., Rajvanshy, H., Sirigiri, A., Gregg, J.M., Lin, S., Martinez, G.J., Mattingly, S.M., Mirjafari, S., Mulukutla, R., Nepal, S., Nies, K., Reddy, M.D., Robles-Granda, P., Campbell, A.T., Chawla, N.V., D’Mello, S., Dey, A.K., Jiang, K., Liu, Q., Mark, G., Moskal, E., Striegel, A., Tay, L., Abowd, G.D., De Choudhury, M.: A multisensor person-centered approach to understand the role of daily activities in job performance with organizational personas. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3(4) (2019)

    Google Scholar 

  3. Wang, W., Mirjafari, S., Harari, G., Ben-Zeev, D., Brian, R., Choudhury, T., Hauser, M., Kane, J., Masaba, K., Nepal, S., Sano, A., Scherer, E., Tseng, V., Wang, R., Wen, H., Jialing, W., Campbell, A.: Social Sensing: Assessing Social Functioning of Patients Living with Schizophrenia Using Mobile Phone Sensing, pp. 1–15. Association for Computing Machinery, New York, NY, USA (2020)

    Google Scholar 

  4. Yang, F., Han, T., Deng, K., Han, Y.: The application of artificial intelligence in the mental diseases. In: Proceedings of the 2020 Conference on Artificial Intelligence and Healthcare, CAIH2020, pp. 36–40. Association for Computing Machinery, New York, NY, USA (2020)

    Google Scholar 

  5. Labor Ministry of Health and Welfare: Overview of the 2018 Occupational Safety and Health Survey (Fact-Finding Survey). https://www.mhlw.go.jp/toukei/list/dl/h30-46-50_gaikyo.pdf

  6. Kotteeswari, M., Sharief, S.T.: Job Stress and Its Impact on Employees’ Performance a Study with Reference to Employees Working in BPOS (2014)

    Google Scholar 

  7. Warr, P., Nielsen, K.: Wellbeing and Work Performance, 02 2018

    Google Scholar 

  8. Kopp, M.S., Stauder, A., Purebl, G., Janszky, I., Skrabski, A.: Work stress and mental health in a changing society. Eur. J. Public Health 18(3), 238–244 (2007)

    Google Scholar 

  9. Yu, H., Itoh, A., Sakamoto, R., Shimaoka, M., Sano, A.: Forecasting Health and Wellbeing for Shift Workers Using Job-Role Based Deep Neural Network, pp. 89–103, 02 2021

    Google Scholar 

  10. Spathis, D., Servia-Rodriguez, S., Farrahi, K., Mascolo, C., Rentfrow, J.: Sequence multi-task learning to forecast mental wellbeing from sparse self-reported data. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery. Data Mining, KDD’19, pp. 2886–2894. Association for Computing Machinery, New York, NY, USA (2019)

    Google Scholar 

  11. Sano, A., Picard, R.W.: Stress recognition using wearable sensors and mobile phones. In: Proceedings of the 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, ACII’13, pp. 671–676. IEEE Computer Society, USA (2013)

    Google Scholar 

  12. Koldijk, S., Neerincx, M.A., Kraaij, W.: Detecting work stress in offices by combining unobtrusive sensors. IEEE Trans. Affect. Comput. 9(02), 227–239 (2018)

    Article  Google Scholar 

  13. Alberdia, A., Aztiriaa, A., Basarabb, A., Cook, D.J.: Using Smart Offices to Predict Occupational Stress

    Google Scholar 

  14. Sano, A., Phillips, A., Yu, A.Z., McHill, A.W., Taylor, S., Jaques, N., Czeisler, C., Klerman, E., Picard, R.W.: Recognizing academic performance, sleep quality, stress level, and mental health using personality traits, wearable sensors and mobile phones. In: 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 1–6 (2015)

    Google Scholar 

  15. Robles-Granda, P., Lin, S, Wu, X., D’Mello, S., Martínez, G.J., Saha, K., Nies, K., Mark, G., Campbell, A.T., De Choudhury, M., Dey, A.D., Gregg, J.M., Grover, T., Mattingly, S.M., Mirjafari, S., Moskal, E., Striegel, A., Chawla, N.V.: Jointly predicting job performance, personality, cognitive ability, affect, and well-being. CoRR (2020). abs/2006.08364

    Google Scholar 

  16. Rahmani, A.M., Nejad, N.T., Perego, P.: A method for simplified HRQOL measurement by smart devices. In: Wireless Mobile Communication and Healthcare, pp. 91–98 (2017)

    Google Scholar 

  17. Feng, T., Booth, B., Baldwin-Rodríguez, B., Osorno, F., Narayanan, S.: A multimodal analysis of physical activity, sleep, and work shift in nurses with wearable sensor data. Sci. Rep. 11, 04 2021

    Google Scholar 

  18. Lee, M.: Detecting affective flow states of knowledge workers using physiological sensors. CoRR (2020). abs/2006.10635

    Google Scholar 

  19. Fukuda, H., Tani, Y., Matsuda, H., Arakawa, Y., Yasumoto, K.: An analysis of the relationship between office workers’ sleep status and occupational health indicators. Technical Report 22, Nara Institute of Science and Technology, Kyushu University/JST PRESTO, Nov 2019

    Google Scholar 

  20. Garcia-Ceja, E., Osmani, V., Mayora, O.: Automatic stress detection in working environments from smartphones’ accelerometer data: a first step. IEEE J. Biomed. Health Inform. 20(4), 1053–1060 (2016)

    Article  Google Scholar 

  21. Zenonos, A., Khan, A., Kalogridis, G., Vatsikas, S., Lewis, T., Sooriyabandara, M.: HealthyOffice: Mood Recognition at Work Using Smartphones and Wearable Sensors, pp. 1–6, 03 2016

    Google Scholar 

  22. Mirjafari, S., Masaba, K., Grover, T., Wang, W., Audia, P., Campbell, A.T., Chawla, N.V., Das Swain, V., De Choudhury, M., Dey, A.K., D’Mello, S.K., Gao, G., Gregg, J.M., Jagannath, K., Jiang, K., Lin, S., Liu, Q., Mark, G., Martinez, G.J., Mattingly, S.M., Moskal, E., Mulukutla, R., Nepal, S., Nies, K., Reddy, M.D., Robles-Granda, P., Saha, K., Sirigiri, A., Striegel, A.: Differentiating higher and lower job performers in the workplace using mobile sensing. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3(2), June 2019

    Google Scholar 

  23. Sano, A., Taylor, S., Mchill, A., Phillips, A., Barger, L., Klerman, E., Picard, R.: Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones. J. Med. Internet Res. 20, 11 (2017)

    Google Scholar 

  24. Suhara, Y., Xu, Y., ‘Sandy’ Pentland, A.: DeepMood: forecasting depressed mood based on self-reported histories via recurrent neural networks. In: Proceedings of the 26th International Conference on World Wide Web, WWW’17, pp. 715–724. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE (2017)

    Google Scholar 

  25. Umematsu, T., Sano, A., Taylor, S., Tsujikawa, M., Picard, R.W.: Forecasting stress, mood, and health from daytime physiology in office workers and students. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC), pp. 5953–5957 (2020)

    Google Scholar 

  26. Lutchyn, Y., Johns, P., Czerwinski, M., Iqbal, S., Mark, G., Sano, A.: Stress is in the eye of the beholder. In: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 119–124 (2015)

    Google Scholar 

  27. Shuck, B., Reio, T.: Employee engagement and well-being. J. Leadersh. Organ. Stud. 21, 43–58 (2013)

    Google Scholar 

  28. Matsuda, Y., Inoue, S., Tani, Y., Fukuda, S., Arakawa, Y.: WorkerSense: mobile sensing platform for collecting physiological, mental, and environmental state of office workers. In: 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) (2020)

    Google Scholar 

  29. Inoue, S., Lago, P., Hossain, T., Mairittha, T., Mairittha, N.: Integrating activity recognition and nursing care records: the system, deployment, and a verification study. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3(3) (2019)

    Google Scholar 

  30. Fukuda, I.: Attempting to develop depression and anxiety mood scale (DAMS). Action Therapy Res. 23(2), 83–93 (1997)

    Google Scholar 

  31. Kubo, T., Joh, N., Takeyama, H., Makihara, T., Inoue, T., Takanishi, T., Aragomo, Y., Murazaki, M., Tetsu, I.: Examination of the Expression Pattern of Fatigue During Consecutive Night Shifts by

    Google Scholar 

  32. Shimazu, A., Sonnentag, S., Kubota, K., Kawakami, N.: Validation of the Japanese version of the recovery experience questionnaire. J. Occup. Health 54, 03 (2012)

    Article  Google Scholar 

  33. Barber, C., Arne, B., Berglund, P., Cleary, P.D., McKenas, D., Pronk, N., Simon, G., Stang, P., Ustun, T.B., Wang, P., Kessler, R.C.: The world health organization health and work performance questionnaire (HPQ). J. Occup. Environ. Med. 45, 156–174 (2003)

    Google Scholar 

  34. Taylor, S., Jaques, N., Nosakhare, E., Sano, A., Picard, R.: Personalized multitask learning for predicting tomorrow’s mood, stress, and health. IEEE Trans. Affect. Comput. 11(2), 200–213 (2020)

    Article  Google Scholar 

  35. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, O., Liu, T.-Y.: LightGBM: a highly efficient gradient boosting decision tree. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates Inc. (2017)

    Google Scholar 

  36. Reis, J.C.S., Correia, A., Murai, F., Veloso, A., Benevenuto, F.: Explainable machine learning for fake news detection. In: Proceedings of the 10th ACM Conference on Web Science, WebSci’19, pp. 17–26. Association for Computing Machinery, New York, NY, USA (2019)

    Google Scholar 

  37. Bahador Parsa, A., Movahedi, A., Taghipour, H., Derrible, S., (Kouros) Mohammadian, A.: Toward safer highways, application of XGBoost and SHAP for real-time accident detection and feature analysis. Accid. Anal. Prev. 136, 105405 (2020)

    Google Scholar 

  38. Lundberg, S., Lee, S.-I.: A unified approach to interpreting model predictions. CoRR (2017). abs/1705.07874

    Google Scholar 

  39. Lundberg, S.M., Erion, G.G., Lee, S.-I.: Consistent individualized feature attribution for tree ensembles (2019)

    Google Scholar 

  40. Kluger, A.N.: Commute variability and strain. J. Organ. Behav. 19(2), 147–165 (1998)

    Article  Google Scholar 

  41. Moorman, R.H.: The influence of cognitive and affective based job satisfaction measures on the relationship between satisfaction and organizational citizenship behavior. Human Relations 46(6), 759–776 (1993)

    Article  Google Scholar 

  42. Thieme, A., Belgrave, D., Doherty, G.: Machine learning in mental health: a systematic review of the HCI literature to support the development of effective and implementable ml systems. ACM Trans. Comput.-Hum. Interact. 27(5), Aug 2020

    Google Scholar 

Download references

Acknowledgements

The experiment of this research was carried out in collaboration with companies and universities participating in the “2020 Sensing & Transformation Study Group”, whose secretariat is the Applied Brain Science Consortium.

Author information

Authors and Affiliations

  1. Kyushu Institute of Technology, Fukuoka, Japan

    Yusuke Nishimura, Tahera Hossain & Sozo Inoue

  2. Rice University, Houston, TX, USA

    Akane Sano

  3. NTT Data Institute of Management Consulting, Inc., Tokyo, Japan

    Shota Isomura

  4. Kyushu University, Fukuoka, Japan

    Yutaka Arakawa

Authors
  1. Yusuke Nishimura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tahera Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Akane Sano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shota Isomura
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yutaka Arakawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sozo Inoue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yusuke Nishimura .

Editor information

Editors and Affiliations

  1. University of East London, London, UK

    Md Atiqur Rahman Ahad

  2. Kyushu Institute of Technology, Fukuoka, Japan

    Sozo Inoue

  3. Sussex University, Brighton, UK

    Daniel Roggen

  4. Tokyo University of Agriculture and Technology, Koganei, Tokyo, Japan

    Kaori Fujinami

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nishimura, Y., Hossain, T., Sano, A., Isomura, S., Arakawa, Y., Inoue, S. (2022). Toward the Analysis of Office Workers’ Mental Indicators Based on Wearable, Work Activity, and Weather Data. In: Ahad, M.A.R., Inoue, S., Roggen, D., Fujinami, K. (eds) Sensor- and Video-Based Activity and Behavior Computing. Smart Innovation, Systems and Technologies, vol 291. Springer, Singapore. https://doi.org/10.1007/978-981-19-0361-8_1

Download citation

Publish with us

Policies and ethics

Search

Navigation