Takahiro Hashizume
Koji Yatani
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Recent commodity smartphones have biometric sensing capabilities, allowing their daily use for authentication and identification. This increasing use of biometric systems motivates us to design an opportunistic way to sense user's additional physiological or behavioral data. We define this concurrent physiological or behavioral data sensing during biometric authentication or identification as dual-purpose biometrics. As an instance of dual-purpose biometrics, we develop photoplethysmography (PPG) sensing during mobile fingerprint authentication, called Auth ‘n’ Scan. Our system opportunistically extracts cardiovascular information, such as a heart rate and its variability, while users perform phone unlock of a smartphone. To achieve this sensing, our Auth ‘n’ Scan system attaches four PPG units around a fingerprint sensor. The system also performs noise removal and signal selection to accurately estimate cardiovascular information. This paper presents the hardware implementation and signal processing algorithm of our Auth ‘n’ Scan prototype. We also report our system evaluations with 10 participants, showing that, despite a little low precision (a standard deviation of 3--7), estimation of heart rates with high accuracy (under a mean error of 1) is possible from PPG data of five seconds and longer if their baseline information is given. We discuss the feasibility of opportunistic PPG sensing in mobile fingerprint authentication.
Supplemental Material
Available for Download
zip
hashizume.zip (60.4 MB)
Supplemental movie, appendix, image and software files for, Auth ‘n’ Scan: Opportunistic Photoplethysmography in Mobile Fingerprint Authentication
- Rajendra Acharya U, N Kannathal, and S M Krishnan. 2004. Comprehensive analysis of cardiac health using heart rate signals. Physiological Measurement 25, 5 (2004), 1139--1151. https://doi.org/10.1088/0967-3334/25/5/005 Google Scholar
Cross Ref
- Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, and Robert C. Miller. 2015. Smart Homes That Monitor Breathing and Heart Rate. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘15). ACM, New York, NY, USA, 837--846. https://doi.org/10.1145/2702123.2702200 Google ScholarDigital Library
- John Allen. 2007. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement 28, 3 (2007), R1--R39. https://doi.org/10.1088/0967-3334/28/3/R01 Google ScholarCross Ref
- Heba Aly and Moustafa Youssef. 2016. Zephyr: Ubiquitous Accurate multi-Sensor Fusion-based Respiratory Rate Estimation Using Smartphones. In Proceedings of the 35th Annual IEEE International Conference on Computer Communications (IEEE INFOCOM ‘16). https://doi.org/10.1109/INFOCOM.2016.7524401 Google ScholarCross Ref
- Apple Insider. 2016. Average iPhone user unlocks device 80 times per day, 89% use Touch ID, Apple says. (April 2016). Retrieved November 10, 2016 from http://appleinsider.com/articles/16/04/19/average-iphone-user-unlocks-device-80-times-per-day-89-use-touch-id-apple-saysGoogle Scholar
- Rasekhar Bhagavatula, Blase Ur, Kevin Iacovino, Su Mon Kywe, Lorrie Faith Cranor, and Marios Savvides. 2015. Biometric Authentication on iPhone and Android: Usability, Perceptions, and Influences on Adoption. In Proceedings of the NDSS Workshop on Usable Security (USEC ‘15). Google ScholarCross Ref
- Edward D. Chan, Michael M. Chan, and Mallory M. Chan. 2013. Pulse oximetry: Understanding its basic principles facilitates appreciation of its limitations. Respiratory Medicine 107, 6 (2013), 789--799. https://doi.org/10.1016/j.rmed.2013.02.004 Google ScholarCross Ref
- Hiroshi Chigira, Masayuki Ihara, Minoru Kobayashi, Akimichi Tanaka, and Tomohiro Tanaka. 2014. Heart Rate Monitoring Through the Surface of a Drinkware. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘14). ACM, New York, NY, USA, 685--689. https://doi.org/10.1145/2632048.2632101 Google ScholarDigital Library
- Alexander De Luca, Alina Hang, Emanuel von Zezschwitz, and Heinrich Hussmann. 2015. I Feel Like I'M Taking Selfies All Day!: Towards Understanding Biometric Authentication on Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘15). ACM, New York, NY, USA, 1411--1414. https://doi.org/10.1145/2702123.2702141 Google ScholarDigital Library
- Anind K. Dey, Katarzyna Wac, Denzil Ferreira, Kevin Tassini, Jin-Hyuk Hong, and Julian Ramos. 2011. Getting Closer: An Empirical Investigation of the Proximity of User to Their Smart Phones. In Proceedings of the 13th International Conference on Ubiquitous Computing (UbiComp ‘11). ACM, New York, NY, USA, 163--172. https://doi.org/10.1145/2030112.2030135 Google ScholarDigital Library
- Rachel Eardley, Anne Roudaut, Steve Gill, and Stephen J. Thompson. 2017. Understanding Grip Shifts: How Form Factors Impact Hand Movements on Mobile Phones. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ‘17). ACM, New York, NY, USA, 4680--4691. https://doi.org/10.1145/3025453.3025835 Google ScholarDigital Library
- Biyi Fang, Nicholas D. Lane, Mi Zhang, Aidan Boran, and Fahim Kawsar. 2016. BodyScan: Enabling Radio-based Sensing on Wearable Devices for Contactless Activity and Vital Sign Monitoring. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys ‘16). ACM, New York, NY, USA, 97--110. https://doi.org/10.1145/2906388.2906411 Google ScholarDigital Library
- Erin Griffiths, T. Scott Saponas, and A. J. Bernheim Brush. 2014. Health Chair: Implicitly Sensing Heart and Respiratory Rate. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘14). ACM, New York, NY, USA, 661--671. https://doi.org/10.1145/2632048.2632099 Google ScholarDigital Library
- Christian Holz and Patrick Baudisch. 2010. The Generalized Perceived Input Point Model and How to Double Touch Accuracy by Extracting Fingerprints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘10). ACM, New York, NY, USA, 581--590. https://doi.org/10.1145/1753326.1753413 Google ScholarDigital Library
- Christian Holz and Patrick Baudisch. 2013. Fiberio: A Touchscreen That Senses Fingerprints. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ‘13). ACM, New York, NY, USA, 41--50. https://doi.org/10.1145/2501988.2502021 Google ScholarDigital Library
- Christian Holz and Frank R. Bentley. 2016. On-Demand Biometrics: Fast Cross-Device Authentication. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 3761--3766. https://doi.org/10.1145/2858036.2858139 Google ScholarDigital Library
- Christian Holz and Edward J. Wang. 2017. Glabella: Continuously Sensing Blood Pressure Behavior Using an Unobtrusive Wearable Device. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 58 (Sept. 2017), 23 pages. https://doi.org/10.1145/3132024 Google ScholarDigital Library
- Gary Hsieh, Ian Li, Anind Dey, Jodi Forlizzi, and Scott E. Hudson. 2008. Using Visualizations to Increase Compliance in Experience Sampling. In Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp ‘08). ACM, New York, NY, USA, 164--167. https://doi.org/10.1145/1409635.1409657 Google ScholarDigital Library
- Anil K. Jain, Jianjiang Feng, and Karthik Nandakumar. 2010. Fingerprint Matching. Computer 43, 2 (Feb. 2010), 36--44. https://doi.org/10.1109/MC.2010.38 Google ScholarDigital Library
- P. W. Kamen and A. M. Tonkin. 1995. Application of the Poincaré plot to heart rate variability: a new measure of functional status in heart failure. Australian and New Zealand Journal of Medicine 25, 1 (1995), 18--26. https://doi.org/10.1111/j.1445-5994.1995.tb00573.x Google ScholarCross Ref
- Seungwoo Kang, Sungjun Kwon, Chungkuk Yoo, Sangwon Seo, Kwangsuk Park, Junehwa Song, and Youngki Lee. 2014. Sinabro: Opportunistic and Unobtrusive Mobile Electrocardiogram Monitoring System. In Proceedings of the 15th Workshop on Mobile Computing Systems and Applications (HotMobile ‘14). ACM, New York, NY, USA, Article 11, 6 pages. https://doi.org/10.1145/2565585.2565605 Google ScholarDigital Library
- Amy K Karlson, Benjamin B Bederson, and Jose L Contreras-Vidal. 2008. Understanding one-handed use of mobile devices. In Handbook of Research on User Interface Design and Evaluation for Mobile Technology, Joanna Lumsden (Ed.). IGI Global, Hershey, PA, 86--101. https://doi.org/10.4018/978-1-59904-871-0.ch006 Google ScholarCross Ref
- Ilkka Korhonen, Juha Pärkkä, and Mark van Gils. 2003. Health monitoring in the home of the future. IEEE Engineering in Medicine and Biology Magazine 22, 3 (2003), 66--73. https://doi.org/10.1109/MEMB.2003.1213628 Google ScholarCross Ref
- Jennifer R. Kwapisz, Gary M. Weiss, and Samuel A. Moore. 2011. Activity Recognition using Cell Phone Accelerometers. SIGKDD Explor. Newsl. 12, 2 (March 2011), 74--82. https://doi.org/10.1145/1964897.1964918 Google ScholarDigital Library
- Kristof van Laerhoven, Marko Borazio, and Jan Hendrik Burdinski. 2015. Wear is Your Mobile? Investigating Phone Carrying and Use Habits with a Wearable Device. Frontiers in ICT 2 (2015), 10. https://doi.org/10.3389/fict.2015.00010 Google ScholarCross Ref
- Reham Mohamed and Moustafa Youssef. 2017. HeartSense: Ubiquitous Accurate Multi-Modal Fusion-based Heart Rate Estimation Using Smartphones. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 97 (Sept. 2017), 18 pages. https://doi.org/10.1145/3132028 Google ScholarDigital Library
- Joel Murphy and Yury Gitman. 2011. Open Hardware - World Famous Electronics llc. (2011). Retrieved November 28, 2016 from http://pulsesensor.com/pages/open-hardwareGoogle Scholar
- Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver. 2015. When Attention is Not Scarce - Detecting Boredom from Mobile Phone Usage. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘15). ACM, New York, NY, USA, 825--836. https://doi.org/10.1145/2750858.2804252 Google ScholarDigital Library
- Ming-Zher Poh, Kyunghee Kim, Andrew D. Goessling, Nicholas C. Swenson, and Rosalind W. Picard. 2012. Cardiovascular Monitoring Using Earphones and a Mobile Device. IEEE Pervasive Computing 11, 4 (Oct. 2012), 18--26. https://doi.org/10.1109/MPRV.2010.91 Google ScholarDigital Library
- Ming-Zher Poh, Daniel McDuff, and Rosalind W. Picard. 2011. A Medical Mirror for Non-contact Health Monitoring. In ACM SIGGRAPH 2011 Emerging Technologies (SIGGRAPH ‘11). ACM, New York, NY, USA, Article 2, 1 pages. https://doi.org/10.1145/2048259.2048261 Google ScholarDigital Library
- Lunji Qiu. 2014. Fingerprint sensor technology. In Proceedings of the 9th IEEE Conference on Industrial Electronics and Applications (ICIEA ‘14). IEEE, 1433--1436. https://doi.org/10.1109/ICIEA.2014.6931393 Google ScholarCross Ref
- Nalini K. Ratha, Jonathan H. Connell, and Ruud M. Bolle. 2001. Enhancing security and privacy in biometrics-based authentication systems. IBM Systems Journal 40, 3 (2001), 614--634. https://doi.org/10.1147/sj.403.0614 Google ScholarDigital Library
- Arun Ross and Anil K. Jain. 2004. Multimodal biometrics: An overview. In Proceedings of the 12th European Signal Processing Conference (EUSIPCO ‘04). IEEE, 1221--1224.Google Scholar
- Masoud Rostami, Ari Juels, and Farinaz Koushanfar. 2013. Heart-to-heart (H2H): authentication for implanted medical devices. In Proceedings of the 2013 ACM SIGSAC conference on Computer 8 communications security (CCS ‘13). ACM, New York, NY, USA, 1099--1112. https://doi.org/10.1145/2508859.2516658 Google ScholarDigital Library
- Stefan Sammito, Wiebke Sammito, and Irina Böckelmann. 2016. The circadian rhythm of heart rate variability. Biological Rhythm Research 47, 5 (2016), 717--730. https://doi.org/10.1080/09291016.2016.1183887 Google ScholarCross Ref
- M. Angela Sasse and Ivan Flechais. 2005. Usable security: Why do we need it? How do we get it? In Security and Usability: Designing Secure Systems that People Can Use, Lorrie Faith Cranor and Simson Garfinkel (Eds.). O'Reilly, Chapter 2, 13--30.Google Scholar
- Felix Scholkmann, Jens Boss, and Martin Wolf. 2012. An Efficient Algorithm for Automatic Peak Detection in Noisy Periodic and Quasi-Periodic Signals. Algorithms 5, 4 (2012), 588--603. https://doi.org/10.3390/a5040588 Google ScholarCross Ref
- N. Selvaraj, A. Jaryal, J. Santhosh, K. K. Deepak, and S. Anand. 2008. Assessment of heart rate variability derived from finger-tip photoplethysmography as compared to electrocardiography. Journal of Medical Engineering 8 Technology 32, 6 (2008), 479--484. https://doi.org/10.1080/03091900701781317 Google ScholarCross Ref
- Dale R. Setlak. 2004. Advances in Fingerprint Sensors Using RF Imaging Techniques. In Automatic Fingerprint Recognition Systems, Nalini Ratha and Ruud Bolle (Eds.). Springer, New York, NY, 27--53. https://doi.org/10.1007/0-387-21685-5_2 Google ScholarCross Ref
- Mridula Singh, Abhishek Kumar, Kuldeep Yadav, Himanshu Madhu, and Tridib Mukherjee. 2015. Mauka-mauka: Measuring and Predicting Opportunities for Webcam-based Heart Rate Sensing in Workplace Environment. In Proceedings of the 10th EAI International Conference on Body Area Networks (BodyNets ‘15). ICST, Brussels, Belgium, 96--102. https://doi.org/10.4108/eai.28-9-2015.2261492 Google ScholarDigital Library
- Atsushi Sugiura and Yoshiyuki Koseki. 1998. A User Interface Using Fingerprint Recognition: Holding Commands and Data Objects on Fingers. In Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology (UIST ‘98). ACM, New York, NY, USA, 71--79. https://doi.org/10.1145/288392.288575 Google ScholarDigital Library
- Khai N. Truong, Thariq Shihipar, and Daniel J. Wigdor. 2014. Slide to X: Unlocking the Potential of Smartphone Unlocking. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 3635--3644. https://doi.org/10.1145/2556288.2557044 Google ScholarDigital Library
- Rajan Vaish, Keith Wyngarden, Jingshu Chen, Brandon Cheung, and Michael S. Bernstein. 2014. Twitch Crowdsourcing: Crowd Contributions in Short Bursts of Time. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 3645--3654. https://doi.org/10.1145/2556288.2556996 Google ScholarDigital Library
- Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, and William Freeman. 2012. Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Trans. Graph. 31, 4, Article 65 (July 2012), 8 pages. https://doi.org/10.1145/2185520.2185561 Google ScholarDigital Library
- Takahiro Yoshizaki, Yukari Kawano, Yuki Tada, Azumi Hida, Toru Midorikawa, Kohe Hasegawa, Takeshi Mitani, Taiki Komatsu, and Togo Fumiharu. 2013. Diurnal 24-Hour Rhythm in Ambulatory Heart Rate Variability during the Day Shift in Rotating Shift Workers. Journal of Biological Rhythms 28, 3 (2013), 227--236. https://doi.org/10.1177/0748730413489957 Google ScholarCross Ref
Index Terms
- Auth ‘n’ Scan: Opportunistic Photoplethysmography in Mobile Fingerprint Authentication
Recommendations
A new fibre optic pulse oximeter probe for monitoring splanchnic organ arterial blood oxygen saturation
A new, continuous method of monitoring splanchnic organ oxygen saturation (SpO2) would make the early detection of inadequate tissue oxygenation feasible, reducing the risk of hypoperfusion, severe ischaemia, and, ultimately, death. In an attempt to ...
Classification of blood pressure in critically ill patients using photoplethysmography and machine learning
Highlights- Estimation of blood pressure values using pulse rate variability features shows promise for the continuous, non-invasive measurement of systolic, diastolic, ...
AbstractObjective: The aim of this study was to evaluate the capability of features extracted from photoplethysmography (PPG) based Pulse Rate Variability (PRV) to classify hypertensive, normotensive and hypotensive events, and ...
Assessment of Endothelial Function Using Arterial Pressure Signals
Peripheral arterial tonometry (PAT) is a useful method to assess endothelial function. Analysis of pulse wave amplitude with PAT during reactive hyperemia can be used to study peripheral vascular endothelial function very well. We had injected a ...
Comments