Skip to main content
SpringerLink
Log in

Virtual Safety Assistant: An Efficient Tool for Ensuring Safety During Covid-19 Pandemic

  • Conference paper
  • First Online:
Human-Computer Interaction. User Experience and Behavior (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13304))

Included in the following conference series:

  • 2017 Accesses

Abstract

Using intelligent virtual assistants for controlling employee population in workspaces is a research area that remains unexplored. This paper presents a novel application of virtual humans to enforce Covid-19 safety measures in a corporate workplace. For this purpose, we develop a virtual assistant platform, Chloe, equipped with automatic temperature sensing, facial recognition, and dedicated chatbots to act as an initial filter for ensuring public health. Whilst providing an engaging user interaction experience, Chloe minimizes human to human contact, thus reducing the spread of infectious diseases. Chloe restricts the employee population within the office to government-approved safety norms. We experimented with Chloe as a virtual safety assistant in a company, where she interacted and screened the employees for Covid-19 symptoms. Participants filled an online survey to quantify Chloe’s performance in terms of interactivity, system latency, engagement, and accuracy, for which we received positive feedback. We performed statistical analysis on the survey results that reveal positive results and show effectiveness of Chloe in such applications. We detail system architecture, results and limitations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://docs.unity3d.com/Packages/com.unity.render-pipelines.high-definition@13.0/manual/index.html.

  2. 2.

    Twilio API setup: https://www.twilio.com/.

  3. 3.

    Google cloud console setup: https://console.cloud.google.com.

References

  1. Babu, S., Schmugge, S., Barnes, T., Hodges, L.F.: What would you like to talk about? An evaluation of social conversations with a virtual receptionist. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 169–180. Springer, Heidelberg (2006). https://doi.org/10.1007/11821830_14

    Chapter  Google Scholar 

  2. Babu, S., Schmugge, S., Inugala, R., Rao, S., Barnes, T., Hodges, L.F.: Marve: a prototype virtual human interface framework for studying human-virtual human interaction. In: Panayiotopoulos, T., Gratch, J., Aylett, R., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 120–133. Springer, Heidelberg (2005). https://doi.org/10.1007/11550617_11

    Chapter  Google Scholar 

  3. Cairco, L., Wilson, D.M., Fowler, V., LeBlanc, M.: Avari: animated virtual agent retrieving information. In: Proceedings of the 47th Annual Southeast Regional Conference, pp. 1–6 (2009)

    Google Scholar 

  4. Cassell, J.: Embodied conversational interface agents. Commun. ACM 43(4), 70–78 (2000). https://doi.org/10.1145/332051.332075. https://doi.org/10.1145/332051.332075

  5. Cassell, J., et al.: Mack: Media lab autonomous conversational kiosk. In: Proceedings of IMAGINA, January 2002

    Google Scholar 

  6. Chen, S., Liu, Y., Gao, X., Han, Z.: Mobilefacenets: efficient CNNs for accurate real-time face verification on mobile devices. CoRR abs/1804.07573 (2018). http://arxiv.org/abs/1804.07573

  7. (CNA), C.N.A.: Ai-driven system that speeds up temperature screening piloted at 2 locations (2020). https://www.channelnewsasia.com/news/singapore/wuhan-virus-coronavirus-temperature-ai-tech-12426160

  8. Connecttome: Rachel (2021). https://connectome.to/

  9. Damer, N., Grebe, J.H., Chen, C., Boutros, F., Kirchbuchner, F., Kuijper, A.: The effect of wearing a mask on face recognition performance: an exploratory study (2020)

    Google Scholar 

  10. DeVault, D., et al.: Simsensei kiosk: a virtual human interviewer for healthcare decision support. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems, pp. 1061–1068. AAMAS 2014, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC (2014)

    Google Scholar 

  11. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (2019)

    Google Scholar 

  12. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding (2019)

    Google Scholar 

  13. Evers, M., Nijholt, A.: Jacob - an animated instruction agent in virtual reality. In: Tan, T., Shi, Y., Gao, W. (eds.) ICMI 2000. LNCS, vol. 1948, pp. 526–533. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-40063-X_69

    Chapter  Google Scholar 

  14. Freud, E., Stajduhar, A., Rosenbaum, R.S., Avidan, G., Ganel, T.: The covid-19 pandemic masks the way people perceive faces. Sci. Rep. 10(22344) (2020). https://doi.org/10.1038/s41598-020-78986-9

  15. Gockley, R., et al.: Designing robots for long-term social interaction, pp. 1338–1343 (2005). https://doi.org/10.1109/IROS.2005.1545303

  16. Ibrahim, R., Zin, Z.M.: Study of automated face recognition system for office door access control application. In: 2011 IEEE 3rd International Conference on Communication Software and Networks, pp. 132–136 (2011). https://doi.org/10.1109/ICCSN.2011.6014865

  17. Libby, C., Ehrenfeld, J.: Facial recognition technology in 2021: masks, bias, and the future of healthcare. J. Med. Syst. 45(4), 1–3 (2021). https://doi.org/10.1007/s10916-021-01723-w

    Article  Google Scholar 

  18. Likert, R.: A technique for the measurement of attitudes. Arch. Psychol. no. 140, [s.n.], New York (1932)

    Google Scholar 

  19. Matsuyama, Y., Bhardwaj, A., Zhao, R., Romero, O., Akoju, S., Cassell, J.: Socially-aware animated intelligent personal assistant agent. In: Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. pp. 224–227 (January 2016). https://doi.org/10.18653/v1/W16-3628

  20. Minds, C.: Cloudia (2021). https://www.en.cloudminds.com/home-new/cloud-robots/cloudia/

  21. Noma, T., Zhao, L., Badler, N.: Design of a virtual human presenter. IEEE Comput. Graphics Appl. 20(4), 79–85 (2000). https://doi.org/10.1109/38.851755

    Article  Google Scholar 

  22. Norouzi, N., et al.: A systematic survey of 15 years of user studies published in the intelligent virtual agents conference. In: Proceedings of the 18th International Conference on Intelligent Virtual Agents, pp. 17–22. IVA 2018, Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3267851.3267901. https://doi.org.remotexs.ntu.edu.sg/10.1145/3267851.3267901

  23. Norris, J.S.: Face Detection and Recognition in Office Environments. Master’s thesis, Thesis (M.Eng.)-Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, The address of the publisher (6 1999), uRI: http://hdl.handle.net/1721.1/80108

  24. Ramanathan, M., Mishra, N., Thalmann, N.M.: Nadine humanoid social robotics platform. In: Gavrilova, M., Chang, J., Thalmann, N.M., Hitzer, E., Ishikawa, H. (eds.) CGI 2019. LNCS, vol. 11542, pp. 490–496. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22514-8_49

    Chapter  Google Scholar 

  25. Matsuyama, Y., Bhardwaj, A., Zhao, R., Romero, O., Akoju, S., Cassell, J.: Socially-aware animated intelligent personal assistant agent. In: Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 224–227, January 2016. https://doi.org/10.18653/v1/W16-3628

  26. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823 (2015). https://doi.org/10.1109/CVPR.2015.7298682

  27. Selvi, K., P.Chitrakala, Jenitha, A.: Face recognition based attendance marking system. Int. J. Comput. Sci. Mobile Comput. 3(2), 337–342 (2014)

    Google Scholar 

  28. Serengil, S.I., Ozpinar, A.: Lightface: A hybrid deep face recognition framework. In: 2020 Innovations in Intelligent Systems and Applications Conference (ASYU). pp. 23–27. IEEE (2020). https://doi.org/10.1109/ASYU50717.2020.9259802

  29. Teledyne FLIR: LWIR micro thermal camera module (2020). https://www.flir.asia/products/lepton/?model=3.5%20Lepton

  30. Teledyne FLIR: LWIR micro thermal camera module (2020). https://www.flir.asia/discover/professional-tools/how-does-emissivity-affect-thermal-imaging/

  31. Thermo Works: Infrared emissivity table (2020), https://www.thermoworks.com/emissivity-table

  32. TwentyBN: Millie (2021). https://20bn.com/

Download references

Author information

Authors and Affiliations

  1. Dex-Lab Pte. Ltd., Singapore, Singapore

    Manoj Ramanathan, Aalind Singh, Arathy Suresh, Daniel Thalmann & Nadia Magnenat-Thalmann

  2. Nanyang Technological University, Singapore, Singapore

    Manoj Ramanathan

  3. EPFL, Lausanne, Switzerland

    Daniel Thalmann

  4. Miralab, University of Geneva, Geneva, Switzerland

    Nadia Magnenat-Thalmann

Authors
  1. Manoj Ramanathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aalind Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arathy Suresh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Thalmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nadia Magnenat-Thalmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manoj Ramanathan .

Editor information

Editors and Affiliations

  1. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ramanathan, M., Singh, A., Suresh, A., Thalmann, D., Magnenat-Thalmann, N. (2022). Virtual Safety Assistant: An Efficient Tool for Ensuring Safety During Covid-19 Pandemic. In: Kurosu, M. (eds) Human-Computer Interaction. User Experience and Behavior. HCII 2022. Lecture Notes in Computer Science, vol 13304. Springer, Cham. https://doi.org/10.1007/978-3-031-05412-9_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05412-9_37

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05411-2

  • Online ISBN: 978-3-031-05412-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation