skip to main content
research-article

Combating the Spread of Coronavirus by Modeling Fomites with Depth Cameras

Published: 04 November 2020 Publication History

Abstract

Coronavirus is thought to spread through close contact from person to person. While it is believed that the primary means of spread is by inhaling respiratory droplets or aersols, it may also be spread by touching inanimate objects such as doorknobs and handrails that have the virus on it ("fomites''). The Centers for Disease Control and Prevention (CDC) therefore recommends individuals maintain "social distance'' of more than six feet between one another. It further notes that an individual may be infected by touching a fomite and then touching their own mouth, nose or possibly their eyes. We propose the use of computer vision techniques to combat the spread of coronavirus by sounding an audible alarm when an individual touches their own face, or when multiple individuals come within six feet of one another or shake hands. We further propose using depth cameras to track where people touch parts of their physical environment throughout the day, and a simple model of disease spread among potential fomites. Projection mapping techniques can be used to display likely fomites in realtime, while headworn augmented reality systems can be used by custodial staff to perform more effective cleaning of surfaces. Such techniques may find application in particularly vulnerable settings such as schools, long-term care facilities and physician offices.

Supplementary Material

MP4 File (v4issa203.mp4)
Supplemental video

References

[1]
Arduino Team. 2020. This pair of Arduino glasses stops you from touching your face. https://blog.arduino.cc/2020/03/ 10/this-pair-of-arduino-glasses-stops-you-from-touching-your-face/. Proc. ACM Hum.-Comput. Interact., Vol. 4, No. ISS, Article 203. Publication date: November 2020. 203:12 Andrew D. Wilson
[2]
Z. Cao, G. Hidalgo Martinez, T. Simon, S. Wei, and Y. A. Sheikh. 2019. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence (2019).
[3]
Centers for Disease Control and Prevention. 2020. CDC updates COVID-19 transmission webpage to clarify information about types of spread. https://www.cdc.gov/media/releases/2020/s0522-cdc-updates-covid-transmission.html.
[4]
Centers for Disease Control and Prevention. 2020. How COVID-19 Spreads. https://www.cdc.gov/coronavirus/2019- ncov/prevent-getting-sick/how-covid-spreads.html.
[5]
Xiang 'Anthony' Chen. 2020. FaceOff: Detecting Face Touching with a Wrist-Worn Accelerometer. arXiv:2008.01769 [cs.HC]
[6]
Zhen-Dong Guo, Zhong-Yi Wang, Shou-Feng Zhang, Xiao Li, Lin Li, Chao Li, Yan Cui, Rui-Bin Fu, Yun-Zhu Dong, Xiang-Yang Chi, et al. 2020. Aerosol and surface distribution of severe acute respiratory syndrome coronavirus 2 in hospital wards, Wuhan, China, 2020. Emerg Infect Dis 26, 7 (2020).
[7]
Otmar Hilliges, David Kim, Shahram Izadi, Malte Weiss, and Andrew Wilson. 2012. HoloDesk: Direct 3d Interactions with a Situated See-through Display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI '12). Association for Computing Machinery, New York, NY, USA, 2421--2430. https: //doi.org/10.1145/2207676.2208405
[8]
Shahram Izadi, Richard A. Newcombe, David Kim, Otmar Hilliges, David Molyneaux, Steve Hodges, Pushmeet Kohli, Jamie Shotton, Andrew J. Davison, and Andrew Fitzgibbon. 2011. KinectFusion: Real-Time Dynamic 3D Surface Reconstruction and Interaction. In ACM SIGGRAPH 2011 Talks (Vancouver, British Columbia, Canada) (SIGGRAPH '11). Association for Computing Machinery, NewYork, NY, USA, Article 23, 1 pages. https://doi.org/10.1145/2037826.2037857
[9]
Brett Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, AndrewWilson, Eyal Ofek, Blair MacIntyre, Nikunj Raghuvanshi, and Lior Shapira. 2014. RoomAlive: Magical Experiences Enabled by Scalable, Adaptive Projector- Camera Units. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (Honolulu, Hawaii, USA) (UIST '14). Association for Computing Machinery, New York, NY, USA, 637--644. https://doi.org/10. 1145/2642918.2647383
[10]
Alicia NM Kraay, Michael AL Hayashi, Nancy Hernandez-Ceron, Ian H Spicknall, Marisa C Eisenberg, Rafael Meza, and Joseph NS Eisenberg. 2018. Fomite-mediated transmission as a sufficient pathway: a comparative analysis across three viral pathogens. BMC infectious diseases 18, 1 (2018), 540.
[11]
Mary Y. Y. Lai, Peter K. C. Cheng, and Wilina W. L. Lim. 2005. Survival of Severe Acute Respiratory Syndrome Coronavirus. Clinical Infectious Diseases 41, 7 (10 2005), e67--e71. https://doi.org/10.1086/433186 arXiv:https://academic.oup.com/cid/article-pdf/41/7/e67/881171/41--7-e67.pdf
[12]
David Lindlbauer and Andy D. Wilson. 2018. Remixed Reality: Manipulating Space and Time in Augmented Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, Article 129, 13 pages. https://doi.org/10.1145/3173574. 3173703
[13]
Nicolai Marquardt, Robert Diaz-Marino, Sebastian Boring, and Saul Greenberg. 2011. The Proximity Toolkit: Prototyping Proxemic Interactions in Ubiquitous Computing Ecologies. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST '11). Association for Computing Machinery, New York, NY, USA, 315--326. https://doi.org/10.1145/2047196.2047238
[14]
Microsoft. 2020. About Azure Kinect Sensor SDK. https://docs.microsoft.com/en-us/azure/kinect-dk/about-sensor-sdk.
[15]
Microsoft. 2020. Azure Kinect DK. https://azure.microsoft.com/en-us/services/kinect-dk/.
[16]
Microsoft. 2020. Quickstart: Set up Azure Kinect body tracking. https://docs.microsoft.com/en-us/azure/kinectdk/ body-sdk-setup.
[17]
Microsoft Research. 2015. RoomAlive Toolkit. https://github.com/Microsoft/RoomAliveToolkit.
[18]
Brian Moore Mike Bodge and Isaac Blankensmith. 2020. https://donottouchyourface.com/.
[19]
MIT Media Lab. 2020. Project Saving Face. https://www.media.mit.edu/projects/saving-face/overview/.
[20]
New York Times. 2020. This 3-D Simulation Shows Why Social Distancing Is So Important. https://www.nytimes. com/interactive/2020/04/14/science/coronavirus-transmission-cough-6-feet-ar-ul.html.
[21]
New York Times. 2020. What's the Risk of Catching Coronavirus From a Surface? https://www.nytimes.com/2020/05/ 28/well/live/whats-the-risk-of-catching-coronavirus-from-a-surface.html.
[22]
Mark Nicas and Daniel Best. 2008. A Study Quantifying the Hand-to-Face Contact Rate and Its Potential Application to Predicting Respiratory Tract Infection. Journal of Occupational and Environmental Hygiene 5, 6 (2008), 347--352. https://doi.org/10.1080/15459620802003896 arXiv:https://doi.org/10.1080/15459620802003896 18357546.
[23]
Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L. Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A. Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, and Shahram Izadi. 2016. Holoportation: Virtual 3D Teleportation in Real-Time. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST '16). Association for Computing Proc. ACM Hum.-Comput. Interact., Vol. 4, No. ISS, Article 203. Publication date: November 2020. Combating the Spread of Coronavirus by Modeling Fomites with Depth Cameras 203:13 Machinery, New York, NY, USA, 741--754. https://doi.org/10.1145/2984511.2984517
[24]
Mark Rober. 2020. How To See Germs Spread (Coronavirus). https://www.youtube.com/watch?v=I5-dI74zxPg.
[25]
Jamie Shotton, Toby Sharp, Alex Kipman, Andrew Fitzgibbon, Mark Finocchio, Andrew Blake, Mat Cook, and Richard Moore. 2013. Real-Time Human Pose Recognition in Parts from Single Depth Images. Commun. ACM 56, 1 (Jan. 2013), 116--124. https://doi.org/10.1145/2398356.2398381
[26]
TechCrunch. 2020. Immutouch wristband buzzed to stop touching your face. https://techcrunch.com/2020/03/09/dontimmutouch/.
[27]
Neeltje van Doremalen, Trenton Bushmaker, Dylan H. Morris, Myndi G. Holbrook, Amandine Gamble, Brandi N. Williamson, Azaibi Tamin, Jennifer L. Harcourt, Natalie J. Thornburg, Susan I. Gerber, James O. Lloyd-Smith, Emmie de Wit, and Vincent J. Munster. 2020. Aerosol and Surface Stability of SARS-CoV-2 as Compared with SARS-CoV-1. New England Journal of Medicine 382, 16 (2020), 1564--1567. https://doi.org/10.1056/NEJMc2004973 arXiv:https://doi.org/10.1056/NEJMc2004973
[28]
Washington Post. 2020. Officials keep warning the public not to touch their faces -- and then do just that. https: //www.youtube.com/watch?v=mA1wqjaeKj0.
[29]
Andrew Wilson, Hrvoje Benko, Shahram Izadi, and Otmar Hilliges. 2012. Steerable Augmented Reality with the Beamatron. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (Cambridge, Massachusetts, USA) (UIST '12). Association for Computing Machinery, New York, NY, USA, 413--422. https://doi.org/ 10.1145/2380116.2380169
[30]
Andrew D. Wilson. 2010. Using a Depth Camera as a Touch Sensor. In ACM International Conference on Interactive Tabletops and Surfaces (Saarbrücken, Germany) (ITS '10). Association for Computing Machinery, New York, NY, USA, 69--72. https://doi.org/10.1145/1936652.1936665
[31]
Andrew D. Wilson. 2017. Fast Lossless Depth Image Compression. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (Brighton, United Kingdom) (ISS '17). Association for Computing Machinery, New York, NY, USA, 100--105. https://doi.org/10.1145/3132272.3134144
[32]
Andrew D. Wilson and Hrvoje Benko. 2010. Combining Multiple Depth Cameras and Projectors for Interactions on, above and between Surfaces. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (New York, New York, USA) (UIST '10). Association for Computing Machinery, New York, NY, USA, 273--282. https://doi.org/10.1145/1866029.1866073
[33]
Chi-Jui Wu, Steven Houben, and Nicolai Marquardt. 2017. EagleSense: Tracking People and Devices in Interactive Spaces Using Real-Time Top-View Depth-Sensing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 3929--3942. https://doi.org/10.1145/3025453.3025562
[34]
Robert Xiao, Scott Hudson, and Chris Harrison. 2016. DIRECT: Making Touch Tracking on Ordinary Surfaces Practical with Hybrid Depth-Infrared Sensing. In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces (Niagara Falls, Ontario, Canada) (ISS '16). Association for Computing Machinery, New York, NY, USA, 85--94. https://doi.org/10.1145/2992154.2992173
[35]
Jijun Zhao, Joseph E Eisenberg, Ian H Spicknall, Sheng Li, and James S Koopman. 2012. Model analysis of fomite mediated influenza transmission. PloS one 7, 12 (2012).

Cited By

View all
  • (2025)Efficient social distance maintenance with augmented realityVirtual Reality10.1007/s10055-025-01102-729:1Online publication date: 20-Jan-2025
  • (2022)Evaluating Automatic Body Orientation Detection for Indoor Location from Skeleton Tracking Data to Detect Socially Occupied Spaces Using the Kinect v2, Azure Kinect and Zed 2iSensors10.3390/s2210379822:10(3798)Online publication date: 17-May-2022
  • (2022)Don’t Touch This! - Investigating the Potential of Visualizing Touched Surfaces on the Consideration of Behavior ChangePersuasive Technology10.1007/978-3-030-98438-0_6(75-88)Online publication date: 29-Mar-2022
  • Show More Cited By

Index Terms

  1. Combating the Spread of Coronavirus by Modeling Fomites with Depth Cameras

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Human-Computer Interaction
    Proceedings of the ACM on Human-Computer Interaction  Volume 4, Issue ISS
    ISS
    November 2020
    488 pages
    EISSN:2573-0142
    DOI:10.1145/3433930
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 04 November 2020
    Published in PACMHCI Volume 4, Issue ISS

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. covid-19
    2. depth cameras, sensing, augmented reality, coronavirus

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)7
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Efficient social distance maintenance with augmented realityVirtual Reality10.1007/s10055-025-01102-729:1Online publication date: 20-Jan-2025
    • (2022)Evaluating Automatic Body Orientation Detection for Indoor Location from Skeleton Tracking Data to Detect Socially Occupied Spaces Using the Kinect v2, Azure Kinect and Zed 2iSensors10.3390/s2210379822:10(3798)Online publication date: 17-May-2022
    • (2022)Don’t Touch This! - Investigating the Potential of Visualizing Touched Surfaces on the Consideration of Behavior ChangePersuasive Technology10.1007/978-3-030-98438-0_6(75-88)Online publication date: 29-Mar-2022
    • (2021)Designing a Multi-Agent Occupant Simulation System to Support Facility Planning and Analysis for COVID-19Proceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462030(15-30)Online publication date: 28-Jun-2021
    • (2021)Simulation and Visualization of Virus Transmission for Architectural Design AnalysisACM SIGGRAPH 2021 Talks10.1145/3450623.3464638(1-2)Online publication date: 31-Jul-2021

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media