Skip to main content
SpringerLink
Log in

Anomalous Pattern Recognition in Vital Health Signals via Multimodal Fusion

  • Conference paper
  • First Online:
Body Area Networks. Smart IoT and Big Data for Intelligent Health Management (BODYNETS 2021)

Included in the following conference series:

  • 642 Accesses

Abstract

Increasingly, caregiving to senior citizens and patients requires monitoring of vital signs like heartbeat, respiration, and blood pressure for an extended period. In this paper, we propose a multimodal synchronized biological signal analysis using a deep neural network-based model that may learn to classify different anomalous patterns. The proposed cepstral-based peak fusion technique is designed to model the robust characterization of each biological signal by combining the list of dominant peaks in the input signal and its corresponding cepstrum. This works as an input to the following multimodal anomaly detection process that not only enables accurate identification and localization of aberrant signal patterns but also facilitates the proposed model to adopt an individual’s unique health characteristics over time. In this work, we use Electrocardiogram (ECG), Femoral Pulse, Photoplethysmogram (PPG), and Body Temperature to monitor an individual’s health condition. In both publicly available datasets as well as our lab-based study with 10 participants, the proposed cepstral-based fusion module attains around 7 to \(10\%\) improvement over the baseline of time-domain analysis and the proposed deep learning classifier reports an average accuracy of \(95.5\%\) with 8 classes and \(93\%\) (improvement of \(3\%\)) with 17 classes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://nrs.harvard.edu/URN-3:HUL.INSTREPOS:37364585.

  2. 2.

    https://cdn.shopify.com/s/files/1/0100/6632/files/PulseSensorAmpd_-_Schematic.pdf?1862089645030619491.

References

  1. Mukhopadhyay, S.C.: Wearable sensors for human activity monitoring: a review. IEEE Sensor. J. 15(3), 1321–1330 (2014)

    Article  Google Scholar 

  2. Fletcher, R., Poh, M., Eydgahi, H.: Wearable sensors: opportunities and challenges for low-cost health care. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), vol. 1766 (1763)

    Google Scholar 

  3. Niswar, M., Nur, M., Ilham, A.A., Mappangara, I.: A low cost wearable medical device for vital signs monitoring in low-resource settings. Int. J. Electr. Comput. Eng. 9, 2088–8708 (2019)

    Google Scholar 

  4. Huang, M.C., Liu, J.J., Xu, W., Gu, C., Li, C., Sarrafzadeh, M.: A self-calibrating radar sensor system for measuring vital signs. IEEE Trans. Biomed. Circuits Syst. 10(2), 352–363 (2016)

    Article  Google Scholar 

  5. Rathore, A.S., Li, Z., Zhu, W., Jin, Z., Xu, W.: A survey on heart biometrics. ACM Comput. Surv. (CSUR) 53(6), 1–38 (2020)

    Article  Google Scholar 

  6. Patil, O.R., Wang, W., Gao, Y., Xu, W., Jin, Z.: A low-cost, camera-based continuous PPG monitoring system using Laplacian pyramid. Smart Health 9—-10, 2–11 (2018). CHASE 2018 Special Issue

    Article  Google Scholar 

  7. Lim, Y.G., et al.: Monitoring physiological signals using nonintrusive sensors installed in daily life equipment. Biomed. Eng. Lett. 1(1), 11–20 (2011). https://doi.org/10.1007/s13534-011-0012-0

    Article  Google Scholar 

  8. Brüser, C., Antink, C.H., Wartzek, T., Walter, M., Leonhardt, S.: Ambient and unobtrusive cardiorespiratory monitoring techniques. IEEE Rev. Biomed. Eng. 8, 30–43 (2015)

    Article  Google Scholar 

  9. Kang, S., et al.: Sinabro: opportunistic and unobtrusive mobile electrocardiogram monitoring system. In: Proceedings of the 15th Workshop on Mobile Computing Systems and Applications, pp. 1–6 (2014)

    Google Scholar 

  10. Malik, A.R., Pilon, L., Boger, J.: Development of a smart seat cushion for heart rate monitoring using ballistocardiography. In: 2019 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), pp. 379–383 IEEE (2019)

    Google Scholar 

  11. Yu, X., et al.: A multi-modal sensor for a bed-integrated unobtrusive vital signs sensing array. IEEE Trans. Biomed. Circuits Syst. 13(3), 529–539 (2019)

    Article  Google Scholar 

  12. Wang, S., et al.: Noninvasive monitoring of vital signs based on highly sensitive fiber optic mattress. IEEE Sens. J. 20(11), 6182–6190 (2020)

    Article  Google Scholar 

  13. Pronk, N.: The problem with too much sitting: a workplace conundrum. ACSM’s Health Fit. J. 15(1), 41–43 (2011)

    Article  Google Scholar 

  14. Arnrich, B., Setz, C., La Marca, R., Tröster, G., Ehlert, U.: What does your chair know about your stress level? IEEE Trans. Inf. Technol. Biomed. 14(2), 207–214 (2009)

    Article  Google Scholar 

  15. Zheng, Y., Morrell, J.B.: A vibrotactile feedback approach to posture guidance. In: 2010 IEEE haptics Symposium, pp. 351–358. IEEE (2010)

    Google Scholar 

  16. Baek, H.J., Chung, G.S., Kim, K.K., Park, K.S.: A smart health monitoring chair for nonintrusive measurement of biological signals. IEEE Trans. Inf. Technol. Biomed. 16(1), 150–158 (2011)

    Article  Google Scholar 

  17. Kumar, R., Bayliff, A., De, D., Evans, A., Das, S.K., Makos, M.: Care-chair: sedentary activities and behavior assessment with smart sensing on chair backrest. In: 2016 IEEE International Conference on Smart Computing (SMARTCOMP), pp. 1–18. IEEE (2016)

    Google Scholar 

  18. Griffiths, E., Saponas, T.S., Brush, A.B.: Health chair: implicitly sensing heart and respiratory rate. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 661–671 (2014)

    Google Scholar 

  19. Ravichandran, R., Saba, E., Chen, K.Y., Goel, M., Gupta, S., Patel, S.N.: WiBreathe: estimating respiration rate using wireless signals in natural settings in the home. In: 2015 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp. 131–139. IEEE (2015)

    Google Scholar 

  20. Zhang, Y., Chen, Z., Chen, W., Li, H.: Unobtrusive and continuous bcg-based human identification using a microbend fiber sensor. IEEE Access 7, 72518–72527 (2019)

    Article  Google Scholar 

  21. Bazi, Y., Alajlan, N., AlHichri, H., Malek, S.: Domain adaptation methods for ECG classification. In: 2013 International Conference on Computer Medical Applications (ICCMA), pp. 1–4 (2013)

    Google Scholar 

  22. Lin, C.C., Yang, C.M.: Heartbeat classification using normalized RR intervals and wavelet features. In: 2014 International Symposium on Computer. Consumer and Control, pp. 650–653 (2014)

    Google Scholar 

  23. Zhang, Z., Luo, X.: Heartbeat classification using decision level fusion. Biomed. Eng. Lett. 4(4), 388–395 (2014). https://doi.org/10.1007/s13534-014-0158-7

    Article  Google Scholar 

  24. Zhang, Z., Dong, J., Luo, X., Choi, K.S., Wu, X.: Heartbeat classification using disease-specific feature selection. Comput. Biol. Med. 46, 79–89 (2014)

    Article  Google Scholar 

  25. Pławiak, P.: Novel methodology of cardiac health recognition based on ECG signals and evolutionary-neural system. Expert Syst. Appl. 92, 334–349 (2018)

    Article  Google Scholar 

  26. Hui, X., Kan, E.C.: Seat integration of RF vital-sign monitoring. In: 2019 IEEE MTT-S International Microwave Biomedical Conference (IMBioC), vol. 1, pp. 1–3. (2019)

    Google Scholar 

  27. Gui, Q., Ruiz-Blondet, M.V., Laszlo, S., Jin, Z.: A survey on brain biometrics. ACM Comput. Surv. (CSUR) 51(6), 1–38 (2019)

    Article  Google Scholar 

  28. Anttonen, J., Surakka, V.: Emotions and heart rate while sitting on a chair. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 491–499 (2005)

    Google Scholar 

  29. Pinto, J.R., Cardoso, J.S., Lourenço, A., Carreiras, C.: Towards a continuous biometric system based on ECG signals acquired on the steering wheel. Sensors 17(10), 2228 (2017)

    Article  Google Scholar 

  30. Sidek, K.A., Mai, V., Khalil, I.: Data mining in mobile ECG based biometric identification. J. Netw. Comput. Appl. 44, 83–91 (2014)

    Article  Google Scholar 

  31. Kavsaoğlu, A.R., Polat, K., Bozkurt, M.R.: A novel feature ranking algorithm for biometric recognition with PPG signals. Comput. Biol. Med. 49, 1–14 (2014)

    Article  Google Scholar 

  32. Li, M., Narayanan, S.: Robust ECG biometrics by fusing temporal and cepstral information. In: 2010 20th International Conference on Pattern Recognition, pp. 1326–1329. IEEE (2010)

    Google Scholar 

  33. Da Silva, H.P., Fred, A., Lourenço, A., Jain, A.K.: Finger ECG signal for user authentication: usability and performance. In: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8. IEEE (2013)

    Google Scholar 

  34. Lin, S.L., Chen, C.K., Lin, C.L., Yang, W.C., Chiang, C.T.: Individual identification based on chaotic electrocardiogram signals during muscular exercise. IET Biometrics 3(4), 257–266 (2014)

    Article  Google Scholar 

  35. Lin, F., Song, C., Zhuang, Y., Xu, W., Li, C., Ren, K.: Cardiac scan: a non-contact and continuous heart-based user authentication system. In: Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, pp. 315–328 (2017)

    Google Scholar 

  36. Gürkan, H., Guz, U., Yarman, B.S.: A novel biometric authentication approach using electrocardiogram signals. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4259–4262. IEEE (2013)

    Google Scholar 

  37. Venkatesh, N., Jayaraman, S.: Human electrocardiogram for biometrics using DTW and FLDA. In: 2010 20th International Conference on Pattern Recognition, pp. 3838–3841. IEEE (2010)

    Google Scholar 

  38. Donida Labati, R., Muñoz, E., Piuri, V., Sassi, R., Scotti, F.: Deep-ECG: convolutional neural networks for ECG biometric recognition. Pattern Recogn. Lett. 126, 78–85 (2019)

    Article  Google Scholar 

  39. Tavallali, P., Razavi, M., Pahlevan, N.: Artificial intelligence estimation of carotid-femoral pulse wave velocity using carotid waveform. Sci. Rep. 8, 1–12 (2018)

    Google Scholar 

  40. Ledezma, C.A., Zhou, X., Rodríguez, B., Tan, P.J., Díaz-Zuccarini, V.: A modeling and machine learning approach to ECG feature engineering for the detection of ischemia using pseudo-ECG. PLOS One 14, 1–21 (2019)

    Article  Google Scholar 

  41. da Silva Luz, E.J., Moreira, G.J., Oliveira, L.S., Schwartz, W.R., Menotti, D.: Learning deep off-the-person heart biometrics representations. IEEE Trans. Inf. Forensics Secur. 13(5), 1258–1270 (2017)

    Article  Google Scholar 

  42. Zhang, Q., Zhou, D., Zeng, X.: HeartID: a multiresolution convolutional neural network for ECG-based biometric human identification in smart health applications. Ieee Access 5, 11805–11816 (2017)

    Article  Google Scholar 

  43. Xiang, Y., Luo, J., Zhu, T., Wang, S., Xiang, X., Meng, J.: ECG-based heartbeat classification using two-level convolutional neural network and RR interval difference. IEICE Trans. Inf. Syst. 101(4), 1189–1198 (2018)

    Article  Google Scholar 

  44. Deshmane, M., Madhe, S.: ECG based biometric human identification using convolutional neural network in smart health applications. In: International Conference on Computing Communication Control and Automation (ICCUBEA), pp. 1–6 (2018)

    Google Scholar 

  45. Salloum, R., Kuo, C.C.J.: ECG-based biometrics using recurrent neural networks. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2062–2066. IEEE (2017)

    Google Scholar 

  46. Everson, L., et al.: BiometricNet: deep learning based biometric identification using wrist-worn PPG. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5. IEEE (2018)

    Google Scholar 

  47. Salloum, R., Kuo, C.J.: ECG-based biometrics using recurrent neural networks. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2062–2066 (2017)

    Google Scholar 

  48. Dezaki, F.T., et al.: Deep residual recurrent neural networks for characterisation of cardiac cycle phase from echocardiograms. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS-2017. LNCS, vol. 10553, pp. 100–108. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_12

    Chapter  Google Scholar 

  49. Hui, X., Conroy, T.B., Kan, E.C.: Multi-point near-field RF sensing of blood pressures and heartbeat dynamics. IEEE Access 8, 89935–89945 (2020)

    Article  Google Scholar 

  50. Sidikova, M., et al.: Vital sign monitoring in car seats based on electrocardiography, ballistocardiography and seismocardiography: A review. Sensors 20(19), 5699 (2020)

    Article  Google Scholar 

  51. Singh, R.K., Sarkar, A., Anoop, C.S.: A health monitoring system using multiple non-contact ECG sensors for automotive drivers. In: 2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings, pp. 1–6 (2016)

    Google Scholar 

  52. Kim, K.K., Lim, Y.K., Park, K.S.: The electrically noncontacting ECG measurement on the toilet seat using the capacitively-coupled insulated electrodes. In: The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 1, pp. 2375–2378. IEEE (2004)

    Google Scholar 

  53. Wu, K.F., Chan, C.H., Zhang, Y.t.: Contactless and cuffless monitoring of blood pressure on a chair using e-textile materials. In: 2006 3rd IEEE/EMBS International Summer School on Medical Devices and Biosensors, pp. 98–100. IEEE (2006)

    Google Scholar 

  54. Choi, M., Jeong, J., Kim, S., Kim, S.: Reduction of motion artifacts and improvement of R peak detecting accuracy using adjacent non-intrusive ECG sensors. Sensors 16, 715 (2016)

    Article  Google Scholar 

  55. Coutinho, D.P., Silva, H., Gamboa, H., Fred, A., Figueiredo, M.: Novel fiducial and non-fiducial approaches to electrocardiogram-based biometric systems. IET Biometrics 2(2), 64–75 (2013)

    Article  Google Scholar 

  56. Adib, F., Mao, H., Kabelac, Z., Katabi, D., Miller, R.C.: Smart homes that monitor breathing and heart rate. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 837–846. Association for Computing Machinery, New York (2015)

    Google Scholar 

  57. Welkowitz, W., Cui, Q., Qi, Y., Kostis, J.B.: Noninvasive estimation of cardiac output. IEEE Trans. Biomed. Eng. 38(11), 1100–1105 (1991)

    Article  Google Scholar 

  58. Smith, J., Camporota, L., Beale, R.: Monitoring arterial blood pressure and cardiac output using central or peripheral arterial pressure waveforms. In: Vincent, J.L. (ed.) Intensive Care Medicine, pp. 285–296. Springer, New York (2009). https://doi.org/10.1007/978-0-387-92278-2_27

    Chapter  Google Scholar 

  59. Chen, J., et al.: High durable, biocompatible, and flexible piezoelectric pulse sensor using single-crystalline III-N thin film. Adv. Funct. Mater. 29, 1903162 (2019)

    Article  Google Scholar 

  60. Winder, S.: Analog and digital filter design. Elsevier, Amsterdam (2002)

    Google Scholar 

  61. Virtanen, P., et al.: SciPy 1.0: Fundamental algorithms for scientific computing in python. Nat. Methods 17(3), 261–272 (2020)

    Google Scholar 

  62. O’Haver, T.: Pragmatic introduction to signal processing (2018)

    Google Scholar 

  63. Richig, J.W., Sleeper, M.M.: Electrocardiography of Laboratory Animals. Academic Press, San Diego (2018)

    Google Scholar 

  64. van Gent, P., Farah, H., Nes, N., Arem, B.: HeartPy: a novel heart rate algorithm for the analysis of noisy signals. Transp. Res. Part F Traffic Psychol. Behav. 66, 368–378 (2019)

    Article  Google Scholar 

  65. Pei, S.C., Lin, H.S.: Minimum-phase FIR filter design using real cepstrum. IEEE Trans. Circuits Syst. II Express Briefs 53(10), 1113–1117 (2006)

    Article  Google Scholar 

  66. Bretscher, O.: Linear Algebra with Applications. Prentice Hall, Eaglewood Cliffs (1997)

    MATH  Google Scholar 

  67. Rogovoy, N., et al.: The dialysis procedure triggers autonomic imbalance and cardiac arrhythmias: insights from continuous 14-day ECG monitoring, April 2019

    Google Scholar 

  68. Fan, J., Upadhye, S., Worster, A.: Understanding receiver operating characteristic (roc) curves. Can. J. Emerg. Med. 8(1), 19–20 (2006)

    Article  Google Scholar 

  69. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  70. Ye, C., Kumar, B.V., Coimbra, M.T.: Heartbeat classification using morphological and dynamic features of ECG signals. IEEE Trans. Biomed. Eng. 59(10), 2930–2941 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Transit Middle School, East Amherst, NY, USA

    Soumyadeep Bhattacharjee

  2. State University of New York at Buffalo, Buffalo, NY, USA

    Huining Li & Wenyao Xu

  3. Gifted Math Program, State University of New York at Buffalo, Buffalo, NY, USA

    Soumyadeep Bhattacharjee

Authors
  1. Soumyadeep Bhattacharjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huining Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenyao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Soumyadeep Bhattacharjee .

Editor information

Editors and Affiliations

  1. University of Glasgow, Glasgow, UK

    Masood Ur Rehman

  2. University of Glasgow, Glasgow, UK

    Ahmed Zoha

Rights and permissions

Reprints and permissions

Copyright information

© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bhattacharjee, S., Li, H., Xu, W. (2022). Anomalous Pattern Recognition in Vital Health Signals via Multimodal Fusion. In: Ur Rehman, M., Zoha, A. (eds) Body Area Networks. Smart IoT and Big Data for Intelligent Health Management. BODYNETS 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 420. Springer, Cham. https://doi.org/10.1007/978-3-030-95593-9_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-95593-9_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-95592-2

  • Online ISBN: 978-3-030-95593-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation