Jonathon Fagert
Skip Abstract Section
Abstract
We present a passive and non-intrusive sensing system for monitoring hand washing activity using structural vibration sensing. Proper hand washing is one of the most effective ways to limit the spread and transmission of disease, and has been especially critical during the COVID-19 pandemic. Prior approaches include direct observation and sensing-based approaches, but are limited in non-clinical settings due to operational restrictions and privacy concerns in sensitive areas such as restrooms. Our work introduces a new sensing modality for hand washing monitoring, which measures hand washing activity-induced vibration responses of sink structures, and uses those responses to monitor the presence and duration of hand washing. Primary research challenges are that vibration responses are similar for different activities, occur on different surfaces/structures, and tend to overlap/coincide. We overcome these challenges by extracting information about signal periodicity for similar activities through cepstrum-based features, leveraging hierarchical learning to differentiate activities on different surfaces, and denoting “primary/secondary” activities based on their relative frequency and importance. We evaluate our approach using real-world hand washing data across four different sink structures/locations, and achieve an average F1-score for hand washing activities of 0.95, which represents an 8.8X and 10.2X reduction in error over two different baseline approaches.
- [1] . 2002. The Illustrated Wavelet Transform Handbook: Introductory Theory and Applications in Science, Engineering, Medicine and Finance. CRC press.Google Scholar
- [2] . 2018. Impact localization in dispersive waveguides based on energy-attenuation of waves with the traveled distance. Mechanical Systems and Signal Processing 105 (2018), 361–376.Google ScholarCross Ref
- [3] . 2012. On quantitative identification of explosion earthquake based on cepstrum computation of HHT and statistical simulation of sub-cluster. In Proceedings of the 31st Chinese Control Conference. IEEE, 5311–5316.Google Scholar
- [4] . 2016. Automated video analysis of handwashing behavior as a potential marker of cognitive health in older adults. IEEE Journal of Biomedical and Health Informatics 20, 2 (
March 2016), 682–690.Google ScholarCross Ref - [5] . 1963. The quefrency analysis of time series for echoes: Cepstrum, pseudo-autocovariance, cross-cepstrum and saphe cracking. Time Series Analysis (1963), 209–243.Google Scholar
- [6] . 2020. OAC: Overlapping office activity classification through IoT-sensed structural vibration. In 2020 IEEE/ACM Fifth International Conference on Internet-of-Things Design and Implementation (IoTDI). IEEE, 216–222.Google Scholar
- [7] . 2019. DeskBuddy: An office activity detection system: Demo abstract. In Proceedings of the 18th International Conference on Information Processing in Sensor Networks. 352–353.Google ScholarDigital Library
- [8] . 2002. Application of a cepstral F statistic for improved depth estimation. Bulletin of the Seismological Society of America 92, 5 (2002), 1675–1693.Google Scholar
- [9] . 2008. Hand hygiene compliance monitoring: Current perspectives from the USA. Journal of Hospital Infection 70 (2008), 2–7.Google Scholar
- [10] . 1979. Applications of digital signal processing, edited by Alan V. Oppenheim.The Journal of the Acoustical Society of America 65, 5 (1979), 1354–1354. Google ScholarCross Ref
- [11] . 2002. Guideline for hand hygiene in health-care settings: Recommendations of the healthcare infection control practices advisory committee and the HICPAC/SHEA/APIC/IDSA hand hygiene task force. Morbidity and Mortality Weekly Report 51, RR-16 (2002), 1–48.Google Scholar
- [12] . 2005. An automatic acoustic bathroom monitoring system. In 2005 IEEE International Symposium on Circuits and Systems. IEEE, 1750–1753.Google Scholar
- [13] . 2000. An Introduction to Support Vector Machines: And Other Kernel-based Learning Methods. Cambridge University Press, New York, NY, USA.Google ScholarCross Ref
- [14] . 2020. MD-Vibe: Physics-informed analysis of patient-induced structural vibration data for monitoring gait health in individuals with muscular dystrophy. In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers. 525–531.Google Scholar
- [15] . 2014. Neuromuscular disease classification based on mel -frequency cepstrum of motor unit action potential. In 2014 International Conference on Electrical Engineering and Information & Communication Technology. IEEE, 1–4.Google Scholar
- [16] . 2019. Occupant-detection strategy using footstep-induced floor vibrations. In Proceedings of the 1st ACM International Workshop on Device-Free Human Sensing. 31–34.Google ScholarDigital Library
- [17] . 2021. Structure- and sampling-adaptive gait balance symmetry estimation using footstep-induced structural floor vibrations. Journal of Engineering Mechanics 147, 2 (2021), 04020151.Google Scholar
- [18] Jonathon Fagert, Mostafa Mirshekari, Shijia Pan, Pei Zhang, and Hae Young Noh. 2017. Characterizing left-right gait balance using footstep-induced structural vibrations. In Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2017, Vol. 10168. 357–365.Google Scholar
- [19] . 2019. Characterizing structural changes to estimate walking gait balance. In Dynamics of Civil Structures, Volume 2. Springer, 333–335.Google Scholar
- [20] . 2019. Gait health monitoring through footstep-induced floor vibrations. In 2019 18th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 319–320.Google Scholar
- [21] . 2020. Structural property guided gait parameter estimation using footstep-induced floor vibrations. In Dynamics of Civil Structures, Volume 2, (Ed.). Springer International Publishing, Cham, 191–194.Google Scholar
- [22] . 2017. Monitoring hand-washing practices using structural vibrations. Structural Health Monitoring (2017).Google Scholar
- [23] . 2019. Vibration source separation for multiple people gait monitoring using footstep-induced floor vibrations. Structural Health Monitoring (2019).Google Scholar
- [24] . 2020. How to Protect Yourself & Others. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.Google Scholar
- [25] . 2001. The Elements of Statistical Learning. Vol. 1. Springer series in statistics New York.Google Scholar
- [26] . 2017. Towards vision-based smart hospitals: A system for tracking and monitoring hand hygiene compliance. In Machine Learning for Healthcare Conference. 75–87.Google Scholar
- [27] . 1993. Automatic language identification using a segment-based approach. In Third European Conference on Speech Communication and Technology.Google Scholar
- [28] . 1990. An automatic means to discriminate between earthquakes and quarry blasts. Bulletin of the Seismological Society of America 80, 6B (1990), 2143–2160.Google Scholar
- [29] I/O Sensor Nederland bv 2006. SM-24 Geophone Element. I/O Sensor Nederland bv.
P/N 1004117 .Google Scholar - [30] . 2000. Fracture source location in thin plates using the wavelet transform of dispersive waves. Ultrasonics, Ferroelectrics, and Frequency Control, IEEE Transactions on 47, 3 (2000), 612–619.Google Scholar
- [31] . 2000. Wavelet analysis of plate wave propagation in composite laminates. Composite Structures 49, 4 (2000), 443–450.Google Scholar
- [32] . 1995. The application of cepstral coefficients and maximum likelihood method in EMG pattern recognition [movements classification]. IEEE Transactions on Biomedical Engineering 42, 8 (1995), 777–785.Google ScholarCross Ref
- [33] . 2019. Vibration-based gait analysis via instrumented buildings. International Journal of Distributed Sensor Networks 15, 10 (2019), 1550147719881608.Google Scholar
- [34] . 2009. Save lives: Clean your hands. A global call for action at the point of care. American Journal of Infection Control 37, 4 (2009), 261–262.Google Scholar
- [35] . 2019. Statistics for Biomedical Engineers and Scientists: How to Visualize and Analyze Data. Academic Press.Google Scholar
- [36] . 2016. Robust occupant detection through step-induced floor vibration by incorporating structural characteristics. In Dynamics of Coupled Structures, Volume 4. Springer, 357–367.Google Scholar
- [37] . 2010. On the analytic wavelet transform. IEEE Transactions on Information Theory 56, 8 (2010), 4135–4156.Google ScholarDigital Library
- [38] . 2014. Research on Kernel Function of Support Vector Machine. Vol. 260. Springer Netherlands, Dordrecht, 827–834.Google Scholar
- [39] . 2016. Benchmark problem for human activity identification using floor vibrations. Expert Systems with Applications 62 (2016), 263–272.Google ScholarDigital Library
- [40] . 2013. A portable trackable hand sanitation device increases hand hygiene. American Journal of Infection Control 41, 6 (2013), S37–S38.Google Scholar
- [41] . 1996. Automatic language identification using large vocabulary continuous speech recognition. In 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, Vol. 2. IEEE, 785–788.Google Scholar
- [42] . 2018. Human gait monitoring using footstep-induced floor vibrations across different structures. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. ACM, 1382–1391.Google ScholarDigital Library
- [43] . 2020. Step-level occupant detection across different structures through footstep-induced floor vibration using model transfer. Journal of Engineering Mechanics 146, 3 (2020), 04019137.Google ScholarCross Ref
- [44] . 2021. Obstruction-invariant occupant localization using footstep-induced structural vibrations. Mechanical Systems and Signal Processing 153 (2021), 107499. Google ScholarCross Ref
- [45] . 2018. Occupant localization using footstep-induced structural vibration. Mechanical Systems and Signal Processing 112 (2018), 77–97.Google ScholarCross Ref
- [46] . 2016. Characterizing wave propagation to improve indoor step-level person localization using floor vibration. In Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2016, Vol. 9803. International Society for Optics and Photonics, 980305.Google Scholar
- [47] . 2015. Harmony: A hand wash monitoring and reminder system using smart watches. In Proceedings of the 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, (Coimbra, Portugal). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), ICST, Brussels, Belgium, 11–20.Google Scholar
- [48] . 2013. Unclean hand detection machine using vision sensor. In 2013 Saudi International Electronics, Communications and Photonics Conference. 1–4.Google Scholar
- [49] . 2004. From frequency to quefrency: A history of the cepstrum. IEEE Signal Processing Magazine 21, 5 (2004), 95–106.Google ScholarCross Ref
- [50] . 2019. Is there a relationship between footstep-impact locations and measured signal characteristics? In Proceedings of the 1st ACM International Workshop on Device-Free Human Sensing. 62–65.Google ScholarDigital Library
- [51] . 2019. Fine-grained recognition of activities of daily living through structural vibration and electrical sensing. In Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation. 149–158.Google ScholarDigital Library
- [52] . 2016. Multiple pedestrian tracking through ambient structural vibration sensing. In SenSys. 366–367.Google Scholar
- [53] . 2018. Characterizing human activity induced impulse and slip-pulse excitations through structural vibration. Journal of Sound and Vibration 414 (2018), 61–80.Google ScholarCross Ref
- [54] . 2017. SurfaceVibe: Vibration-based tap & swipe tracking on ubiquitous surfaces. In 2017 16th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 197–208.Google Scholar
- [55] . 2015. Indoor person identification through footstep induced structural vibration. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. 81–86.Google ScholarDigital Library
- [56] . 2017. FootprintID: Indoor pedestrian identification through ambient structural vibration sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 89.Google Scholar
- [57] . 2009. The World Health Organization guidelines on hand hygiene in health care and their consensus recommendations. Infection Control & Hospital Epidemiology 30, 7 (2009), 611–622.Google ScholarCross Ref
- [58] . 2010. Method for automated monitoring of hand hygiene adherence without radio-frequency identification. Infection Control and Hospital Epidemiology 31, 12 (2010), 1294–1297.Google ScholarCross Ref
- [59] . 2017. A framework for occupancy tracking in a building via structural dynamics sensing of footstep vibrations. Frontiers in Built Environment 3 (2017), 65.Google ScholarCross Ref
- [60] . 2017. Indoor footstep localization from structural dynamics instrumentation. Mechanical Systems and Signal Processing 88 (2017), 224–239.Google ScholarCross Ref
- [61] . 2012. Hand hygiene monitoring goes high-tech. Infection Control Today (2012).Google Scholar
- [62] . 2017. A history of cepstrum analysis and its application to mechanical problems. Mechanical Systems and Signal Processing 97 (2017), 3–19.Google Scholar
- [63] . 2007. Emotion recognition using mel-frequency cepstral coefficients. Information and Media Technologies 2, 3 (2007), 835–848.Google Scholar
- [64] . 2019. Device-free multiple people localization through floor vibration. In Proceedings of the 1st ACM International Workshop on Device-Free Human Sensing. 57–61.Google ScholarDigital Library
- [65] . 2020. O-MedAL: Online active deep learning for medical image analysis. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 10, 4 (2020), e1353.Google Scholar
- [66] . 2018. MedAL: Accurate and robust deep active learning for medical image analysis. In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE, 481–488.Google ScholarCross Ref
- [67] . 1995. Vibration of multi-degree-of-freedom systems with non-proportional viscous damping. International Journal of Mechanical Sciences 37, 4 (1995), 441–455.Google Scholar
- [68] . 1997. Comparison of cepstrum-based methods for radial blind deconvolution of ultrasound images. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 44, 3 (1997), 666–674.Google ScholarCross Ref
- [69] . 1998. Theory of Vibration with Applications. Prentice Hall.
97025851 Google Scholar - [70] . 1995. Adaptive cepstral analysis of speech. IEEE Transactions on Speech and Audio Processing 3, 6 (1995), 481–489.Google Scholar
- [71] . 1993. Application of autoregressive spectral analysis to cepstral estimation of mean scatterer spacing. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 40, 1 (1993), 50–58.Google Scholar
- [72] . 2003. Cepstrum analysis of seismic source characteristics. Acta Seismologica Sinica 16, 1 (2003), 50–58.Google Scholar
- [73] . 2001. Comparison of linear prediction cepstrum coefficients and mel-frequency cepstrum coefficients for language identification. In Proceedings of 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing. ISIMP 2001 (IEEE Cat. No. 01EX489). IEEE, 95–98.Google Scholar
- [74] . 2011. Report on the Burden of Endemic Health Care-Associated Infection Worldwide.
Technical Report .Google Scholar - [75] . 2019. A cepstrum analysis-based classification method for hand movement surface EMG signals. Medical & Biological Engineering & Computing 57, 10 (2019), 2179–2201.Google Scholar
Index Terms
- Clean Vibes: Hand Washing Monitoring Using Structural Vibration Sensing
Recommendations
Fine-Grained Recognition of Activities of Daily Living through Structural Vibration and Electrical Sensing
BuildSys '19: Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and TransportationFine-grained non-intrusive monitoring of activities of daily living (ADL) enables various smart building applications, including ADL pattern assessments for older adults at risk for loss of safety or independence. Prior work in this area has focused on ...
Harmony: A Hand Wash Monitoring and Reminder System using Smart Watches
MOBIQUITOUS'15: proceedings of the 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services on 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and ServicesHand hygiene compliance is extremely important in hospitals, clinics and food businesses. Caregivers' compliance with hand hygiene is one of the most effective tools in preventing healthcare associated infections (HAIs) in hospitals and clinics. In food ...
WashSpot: Real-Time Spotting and Detection of Enacted Compulsive Hand Washing with Wearable Devices
UbiComp/ISWC '22 Adjunct: Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable ComputersThe automatic detection of hand washing has numerous applications in work and medical environments. Checking the compliance with hygiene standard in hospitals, or personal hygiene support are examples thereof. However, hand-washing can also become ...
Comments