skip to main content
research-article

Auth ‘n’ Scan: Opportunistic Photoplethysmography in Mobile Fingerprint Authentication

Published: 08 January 2018 Publication History

Abstract

Recent commodity smartphones have biometric sensing capabilities, allowing their daily use for authentication and identification. This increasing use of biometric systems motivates us to design an opportunistic way to sense user's additional physiological or behavioral data. We define this concurrent physiological or behavioral data sensing during biometric authentication or identification as dual-purpose biometrics. As an instance of dual-purpose biometrics, we develop photoplethysmography (PPG) sensing during mobile fingerprint authentication, called Auth ‘n’ Scan. Our system opportunistically extracts cardiovascular information, such as a heart rate and its variability, while users perform phone unlock of a smartphone. To achieve this sensing, our Auth ‘n’ Scan system attaches four PPG units around a fingerprint sensor. The system also performs noise removal and signal selection to accurately estimate cardiovascular information. This paper presents the hardware implementation and signal processing algorithm of our Auth ‘n’ Scan prototype. We also report our system evaluations with 10 participants, showing that, despite a little low precision (a standard deviation of 3--7), estimation of heart rates with high accuracy (under a mean error of 1) is possible from PPG data of five seconds and longer if their baseline information is given. We discuss the feasibility of opportunistic PPG sensing in mobile fingerprint authentication.

Supplementary Material

hashizume (hashizume.zip)
Supplemental movie, appendix, image and software files for, Auth ‘n’ Scan: Opportunistic Photoplethysmography in Mobile Fingerprint Authentication

References

[1]
Rajendra Acharya U, N Kannathal, and S M Krishnan. 2004. Comprehensive analysis of cardiac health using heart rate signals. Physiological Measurement 25, 5 (2004), 1139--1151. https://doi.org/10.1088/0967-3334/25/5/005
[2]
Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, and Robert C. Miller. 2015. Smart Homes That Monitor Breathing and Heart Rate. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘15). ACM, New York, NY, USA, 837--846. https://doi.org/10.1145/2702123.2702200
[3]
John Allen. 2007. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement 28, 3 (2007), R1--R39. https://doi.org/10.1088/0967-3334/28/3/R01
[4]
Heba Aly and Moustafa Youssef. 2016. Zephyr: Ubiquitous Accurate multi-Sensor Fusion-based Respiratory Rate Estimation Using Smartphones. In Proceedings of the 35th Annual IEEE International Conference on Computer Communications (IEEE INFOCOM ‘16). https://doi.org/10.1109/INFOCOM.2016.7524401
[5]
Apple Insider. 2016. Average iPhone user unlocks device 80 times per day, 89% use Touch ID, Apple says. (April 2016). Retrieved November 10, 2016 from http://appleinsider.com/articles/16/04/19/average-iphone-user-unlocks-device-80-times-per-day-89-use-touch-id-apple-says
[6]
Rasekhar Bhagavatula, Blase Ur, Kevin Iacovino, Su Mon Kywe, Lorrie Faith Cranor, and Marios Savvides. 2015. Biometric Authentication on iPhone and Android: Usability, Perceptions, and Influences on Adoption. In Proceedings of the NDSS Workshop on Usable Security (USEC ‘15).
[7]
Edward D. Chan, Michael M. Chan, and Mallory M. Chan. 2013. Pulse oximetry: Understanding its basic principles facilitates appreciation of its limitations. Respiratory Medicine 107, 6 (2013), 789--799. https://doi.org/10.1016/j.rmed.2013.02.004
[8]
Hiroshi Chigira, Masayuki Ihara, Minoru Kobayashi, Akimichi Tanaka, and Tomohiro Tanaka. 2014. Heart Rate Monitoring Through the Surface of a Drinkware. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘14). ACM, New York, NY, USA, 685--689. https://doi.org/10.1145/2632048.2632101
[9]
Alexander De Luca, Alina Hang, Emanuel von Zezschwitz, and Heinrich Hussmann. 2015. I Feel Like I'M Taking Selfies All Day!: Towards Understanding Biometric Authentication on Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘15). ACM, New York, NY, USA, 1411--1414. https://doi.org/10.1145/2702123.2702141
[10]
Anind K. Dey, Katarzyna Wac, Denzil Ferreira, Kevin Tassini, Jin-Hyuk Hong, and Julian Ramos. 2011. Getting Closer: An Empirical Investigation of the Proximity of User to Their Smart Phones. In Proceedings of the 13th International Conference on Ubiquitous Computing (UbiComp ‘11). ACM, New York, NY, USA, 163--172. https://doi.org/10.1145/2030112.2030135
[11]
Rachel Eardley, Anne Roudaut, Steve Gill, and Stephen J. Thompson. 2017. Understanding Grip Shifts: How Form Factors Impact Hand Movements on Mobile Phones. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ‘17). ACM, New York, NY, USA, 4680--4691. https://doi.org/10.1145/3025453.3025835
[12]
Biyi Fang, Nicholas D. Lane, Mi Zhang, Aidan Boran, and Fahim Kawsar. 2016. BodyScan: Enabling Radio-based Sensing on Wearable Devices for Contactless Activity and Vital Sign Monitoring. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys ‘16). ACM, New York, NY, USA, 97--110. https://doi.org/10.1145/2906388.2906411
[13]
Erin Griffiths, T. Scott Saponas, and A. J. Bernheim Brush. 2014. Health Chair: Implicitly Sensing Heart and Respiratory Rate. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘14). ACM, New York, NY, USA, 661--671. https://doi.org/10.1145/2632048.2632099
[14]
Christian Holz and Patrick Baudisch. 2010. The Generalized Perceived Input Point Model and How to Double Touch Accuracy by Extracting Fingerprints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘10). ACM, New York, NY, USA, 581--590. https://doi.org/10.1145/1753326.1753413
[15]
Christian Holz and Patrick Baudisch. 2013. Fiberio: A Touchscreen That Senses Fingerprints. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ‘13). ACM, New York, NY, USA, 41--50. https://doi.org/10.1145/2501988.2502021
[16]
Christian Holz and Frank R. Bentley. 2016. On-Demand Biometrics: Fast Cross-Device Authentication. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 3761--3766. https://doi.org/10.1145/2858036.2858139
[17]
Christian Holz and Edward J. Wang. 2017. Glabella: Continuously Sensing Blood Pressure Behavior Using an Unobtrusive Wearable Device. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 58 (Sept. 2017), 23 pages. https://doi.org/10.1145/3132024
[18]
Gary Hsieh, Ian Li, Anind Dey, Jodi Forlizzi, and Scott E. Hudson. 2008. Using Visualizations to Increase Compliance in Experience Sampling. In Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp ‘08). ACM, New York, NY, USA, 164--167. https://doi.org/10.1145/1409635.1409657
[19]
Anil K. Jain, Jianjiang Feng, and Karthik Nandakumar. 2010. Fingerprint Matching. Computer 43, 2 (Feb. 2010), 36--44. https://doi.org/10.1109/MC.2010.38
[20]
P. W. Kamen and A. M. Tonkin. 1995. Application of the Poincaré plot to heart rate variability: a new measure of functional status in heart failure. Australian and New Zealand Journal of Medicine 25, 1 (1995), 18--26. https://doi.org/10.1111/j.1445-5994.1995.tb00573.x
[21]
Seungwoo Kang, Sungjun Kwon, Chungkuk Yoo, Sangwon Seo, Kwangsuk Park, Junehwa Song, and Youngki Lee. 2014. Sinabro: Opportunistic and Unobtrusive Mobile Electrocardiogram Monitoring System. In Proceedings of the 15th Workshop on Mobile Computing Systems and Applications (HotMobile ‘14). ACM, New York, NY, USA, Article 11, 6 pages. https://doi.org/10.1145/2565585.2565605
[22]
Amy K Karlson, Benjamin B Bederson, and Jose L Contreras-Vidal. 2008. Understanding one-handed use of mobile devices. In Handbook of Research on User Interface Design and Evaluation for Mobile Technology, Joanna Lumsden (Ed.). IGI Global, Hershey, PA, 86--101. https://doi.org/10.4018/978-1-59904-871-0.ch006
[23]
Ilkka Korhonen, Juha Pärkkä, and Mark van Gils. 2003. Health monitoring in the home of the future. IEEE Engineering in Medicine and Biology Magazine 22, 3 (2003), 66--73. https://doi.org/10.1109/MEMB.2003.1213628
[24]
Jennifer R. Kwapisz, Gary M. Weiss, and Samuel A. Moore. 2011. Activity Recognition using Cell Phone Accelerometers. SIGKDD Explor. Newsl. 12, 2 (March 2011), 74--82. https://doi.org/10.1145/1964897.1964918
[25]
Kristof van Laerhoven, Marko Borazio, and Jan Hendrik Burdinski. 2015. Wear is Your Mobile? Investigating Phone Carrying and Use Habits with a Wearable Device. Frontiers in ICT 2 (2015), 10. https://doi.org/10.3389/fict.2015.00010
[26]
Reham Mohamed and Moustafa Youssef. 2017. HeartSense: Ubiquitous Accurate Multi-Modal Fusion-based Heart Rate Estimation Using Smartphones. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 97 (Sept. 2017), 18 pages. https://doi.org/10.1145/3132028
[27]
Joel Murphy and Yury Gitman. 2011. Open Hardware - World Famous Electronics llc. (2011). Retrieved November 28, 2016 from http://pulsesensor.com/pages/open-hardware
[28]
Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver. 2015. When Attention is Not Scarce - Detecting Boredom from Mobile Phone Usage. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘15). ACM, New York, NY, USA, 825--836. https://doi.org/10.1145/2750858.2804252
[29]
Ming-Zher Poh, Kyunghee Kim, Andrew D. Goessling, Nicholas C. Swenson, and Rosalind W. Picard. 2012. Cardiovascular Monitoring Using Earphones and a Mobile Device. IEEE Pervasive Computing 11, 4 (Oct. 2012), 18--26. https://doi.org/10.1109/MPRV.2010.91
[30]
Ming-Zher Poh, Daniel McDuff, and Rosalind W. Picard. 2011. A Medical Mirror for Non-contact Health Monitoring. In ACM SIGGRAPH 2011 Emerging Technologies (SIGGRAPH ‘11). ACM, New York, NY, USA, Article 2, 1 pages. https://doi.org/10.1145/2048259.2048261
[31]
Lunji Qiu. 2014. Fingerprint sensor technology. In Proceedings of the 9th IEEE Conference on Industrial Electronics and Applications (ICIEA ‘14). IEEE, 1433--1436. https://doi.org/10.1109/ICIEA.2014.6931393
[32]
Nalini K. Ratha, Jonathan H. Connell, and Ruud M. Bolle. 2001. Enhancing security and privacy in biometrics-based authentication systems. IBM Systems Journal 40, 3 (2001), 614--634. https://doi.org/10.1147/sj.403.0614
[33]
Arun Ross and Anil K. Jain. 2004. Multimodal biometrics: An overview. In Proceedings of the 12th European Signal Processing Conference (EUSIPCO ‘04). IEEE, 1221--1224.
[34]
Masoud Rostami, Ari Juels, and Farinaz Koushanfar. 2013. Heart-to-heart (H2H): authentication for implanted medical devices. In Proceedings of the 2013 ACM SIGSAC conference on Computer 8 communications security (CCS ‘13). ACM, New York, NY, USA, 1099--1112. https://doi.org/10.1145/2508859.2516658
[35]
Stefan Sammito, Wiebke Sammito, and Irina Böckelmann. 2016. The circadian rhythm of heart rate variability. Biological Rhythm Research 47, 5 (2016), 717--730. https://doi.org/10.1080/09291016.2016.1183887
[36]
M. Angela Sasse and Ivan Flechais. 2005. Usable security: Why do we need it? How do we get it? In Security and Usability: Designing Secure Systems that People Can Use, Lorrie Faith Cranor and Simson Garfinkel (Eds.). O'Reilly, Chapter 2, 13--30.
[37]
Felix Scholkmann, Jens Boss, and Martin Wolf. 2012. An Efficient Algorithm for Automatic Peak Detection in Noisy Periodic and Quasi-Periodic Signals. Algorithms 5, 4 (2012), 588--603. https://doi.org/10.3390/a5040588
[38]
N. Selvaraj, A. Jaryal, J. Santhosh, K. K. Deepak, and S. Anand. 2008. Assessment of heart rate variability derived from finger-tip photoplethysmography as compared to electrocardiography. Journal of Medical Engineering 8 Technology 32, 6 (2008), 479--484. https://doi.org/10.1080/03091900701781317
[39]
Dale R. Setlak. 2004. Advances in Fingerprint Sensors Using RF Imaging Techniques. In Automatic Fingerprint Recognition Systems, Nalini Ratha and Ruud Bolle (Eds.). Springer, New York, NY, 27--53. https://doi.org/10.1007/0-387-21685-5_2
[40]
Mridula Singh, Abhishek Kumar, Kuldeep Yadav, Himanshu Madhu, and Tridib Mukherjee. 2015. Mauka-mauka: Measuring and Predicting Opportunities for Webcam-based Heart Rate Sensing in Workplace Environment. In Proceedings of the 10th EAI International Conference on Body Area Networks (BodyNets ‘15). ICST, Brussels, Belgium, 96--102. https://doi.org/10.4108/eai.28-9-2015.2261492
[41]
Atsushi Sugiura and Yoshiyuki Koseki. 1998. A User Interface Using Fingerprint Recognition: Holding Commands and Data Objects on Fingers. In Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology (UIST ‘98). ACM, New York, NY, USA, 71--79. https://doi.org/10.1145/288392.288575
[42]
Khai N. Truong, Thariq Shihipar, and Daniel J. Wigdor. 2014. Slide to X: Unlocking the Potential of Smartphone Unlocking. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 3635--3644. https://doi.org/10.1145/2556288.2557044
[43]
Rajan Vaish, Keith Wyngarden, Jingshu Chen, Brandon Cheung, and Michael S. Bernstein. 2014. Twitch Crowdsourcing: Crowd Contributions in Short Bursts of Time. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 3645--3654. https://doi.org/10.1145/2556288.2556996
[44]
Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, and William Freeman. 2012. Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Trans. Graph. 31, 4, Article 65 (July 2012), 8 pages. https://doi.org/10.1145/2185520.2185561
[45]
Takahiro Yoshizaki, Yukari Kawano, Yuki Tada, Azumi Hida, Toru Midorikawa, Kohe Hasegawa, Takeshi Mitani, Taiki Komatsu, and Togo Fumiharu. 2013. Diurnal 24-Hour Rhythm in Ambulatory Heart Rate Variability during the Day Shift in Rotating Shift Workers. Journal of Biological Rhythms 28, 3 (2013), 227--236. https://doi.org/10.1177/0748730413489957

Cited By

View all
  • (2023)EarPPG: Securing Your Identity with Your EarsProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584070(835-849)Online publication date: 27-Mar-2023
  • (2023)LiT: Fine-grained Toothbrushing Monitoring with Commercial LED ToothbrushProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3613287(1-16)Online publication date: 2-Oct-2023
  • (2023)BreathSign: Transparent and Continuous In-ear Authentication Using Bone-conducted Breathing BiometricsIEEE INFOCOM 2023 - IEEE Conference on Computer Communications10.1109/INFOCOM53939.2023.10229037(1-10)Online publication date: 17-May-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 1, Issue 4
December 2017
1298 pages
EISSN:2474-9567
DOI:10.1145/3178157
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 January 2018
Accepted: 01 October 2017
Revised: 01 August 2017
Received: 01 May 2017
Published in IMWUT Volume 1, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Dual-purpose biometrics
  2. cardiovascular sensing
  3. fingerprint authentication
  4. photoplethysmography
  5. unobtrusive sensing

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)15
  • Downloads (Last 6 weeks)1
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2023)EarPPG: Securing Your Identity with Your EarsProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584070(835-849)Online publication date: 27-Mar-2023
  • (2023)LiT: Fine-grained Toothbrushing Monitoring with Commercial LED ToothbrushProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3613287(1-16)Online publication date: 2-Oct-2023
  • (2023)BreathSign: Transparent and Continuous In-ear Authentication Using Bone-conducted Breathing BiometricsIEEE INFOCOM 2023 - IEEE Conference on Computer Communications10.1109/INFOCOM53939.2023.10229037(1-10)Online publication date: 17-May-2023
  • (2023)Unobstrusive smartphone-based oxygen saturation measurement using a Meta-Region of interestPervasive and Mobile Computing10.1016/j.pmcj.2022.10174188:COnline publication date: 1-Jan-2023
  • (2022)SmartPhOx: Smartphone-Based Pulse Oximetry Using a Meta-Region Of Interest2022 IEEE International Conference on Pervasive Computing and Communications (PerCom)10.1109/PerCom53586.2022.9762399(130-140)Online publication date: 21-Mar-2022
  • (2021)Auth+Track: Enabling Authentication Free Interaction on Smartphone by Continuous User TrackingProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445624(1-16)Online publication date: 6-May-2021
  • (2021)HandRate: Heart Rate Monitoring While Simply Holding a Smartphone2021 IEEE International Conference on Pervasive Computing and Communications (PerCom)10.1109/PERCOM50583.2021.9439134(1-11)Online publication date: 22-Mar-2021
  • (2020)Application of Fingerprint-Matching Algorithm in Smart Gun Using Touch-Less Fingerprint Recognition SystemApplications of Artificial Intelligence in Electrical Engineering10.4018/978-1-7998-2718-4.ch007(122-128)Online publication date: 2020
  • (2019)EarEchoProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33512393:3(1-24)Online publication date: 9-Sep-2019
  • (2018)Unlock with Your HeartProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32649502:3(1-22)Online publication date: 18-Sep-2018

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media