Skip to main content
Springer Nature Link
Log in

Multi-Channel Time-Series Person and Soft-Biometric Identification

  • Conference paper
  • First Online:
Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges (ICPR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13643))

Included in the following conference series:

  • 876 Accesses

Abstract

Multi-channel time-series datasets are popular in the context of human activity recognition (HAR). On-body device (OBD) recordings of human movements are often preferred for HAR applications not only for their reliability but as an approach for identity protection, e.g., in industrial settings. Contradictory, the gait activity is a biometric, as the cyclic movement is distinctive and collectable. In addition, the gait cycle has proven to contain soft-biometric information of human groups, such as age and height. Though general human movements have not been considered a biometric, they might contain identity information. This work investigates person and soft-biometrics identification from OBD recordings of humans performing different activities using deep architectures. Furthermore, we propose the use of attribute representation for soft-biometric identification. We evaluate the method on four datasets of multi-channel time-series HAR, measuring the performance of a person and soft-biometrics identification and its relation concerning performed activities. We find that person identification is not limited to gait activity. The impact of activities on the identification performance was found to be training and dataset specific. Soft-biometric based attribute representation shows promising results and emphasis the necessity of larger datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The attribute representation for the two types can be found in https://github.com/nilahnair/Annotation_Tool_LARa/tree/master/From_Human_Pose_to_On_Body_Devices_for_Human_Activity_Recognition/Person_SoftBio_Identification.

References

  1. Anders, C.J., Neumann, D., Samek, W., Müller, K.R., Lapuschkin, S.: Software for dataset-wide XAI: From local explanations to global insights with zennit, CoRelAy, and ViRelAy. http://arxiv.org/abs/2106.13200

  2. Boyd, J.E., Little, J.J.: Biometric gait recognition. In: Tistarelli, M., Bigun, J., Grosso, E. (eds.) Advanced Studies in Biometrics. LNCS, vol. 3161, pp. 19–42. Springer, Heidelberg (2005). https://doi.org/10.1007/11493648_2

    Chapter  Google Scholar 

  3. Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S.T., Tröster, G., Millán, J.d.R., Roggen, D.: The opportunity challenge: A benchmark database for on-body sensor-based activity recognition 34(15), 2033–2042. https://doi.org/10.1016/j.patrec.2012.12.014, https://linkinghub.elsevier.com/retrieve/pii/S0167865512004205

  4. Chunsheng, H., De, W., Huidong, Z., Guoli, L.: Human gait feature data analysis and person identification based on IMU. In: 2020 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), pp. 437–442. IEEE. https://doi.org/10.1109/ICAICA50127.2020.9182691, https://ieeexplore.ieee.org/document/9182691/

  5. Dehzangi, O., Taherisadr, M., ChangalVala, R.: IMU-based gait recognition using convolutional neural networks and multi-sensor fusion 17(12), 2735. https://doi.org/10.3390/s17122735, http://www.mdpi.com/1424-8220/17/12/2735

  6. Elkader, S.A., Barlow, M., Lakshika, E.: Wearable sensors for recognizing individuals undertaking daily activities. In: Proceedings of the 2018 ACM International Symposium on Wearable Computers, pp. 64–67. ACM. https://doi.org/10.1145/3267242.3267245, https://dl.acm.org/doi/10.1145/3267242.3267245

  7. Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1778–1785. IEEE. https://doi.org/10.1109/CVPR.2009.5206772, https://ieeexplore.ieee.org/document/5206772/

  8. Gohar, I., et al.: Person re-identification using deep modeling of temporally correlated inertial motion patterns 20(3), 949. https://doi.org/10.3390/s20030949, https://www.mdpi.com/1424-8220/20/3/949

  9. Grzeszick, R., Lenk, J.M., Rueda, F.M., Fink, G.A., Feldhorst, S., ten Hompel, M.: Deep neural network based human activity recognition for the order picking process. In: Proceedings of the 4th International Workshop on Sensor-Based Activity Recognition and Interaction, pp. 1–6. ACM. https://doi.org/10.1145/3134230.3134231, https://dl.acm.org/doi/10.1145/3134230.3134231

  10. Han, J., Bhanu, B.: Individual recognition using gait energy image 28(2), 316–322. https://doi.org/10.1109/TPAMI.2006.38, https://ieeexplore.ieee.org/document/1561189/

  11. Horst, F., Lapuschkin, S., Samek, W., Müller, K.R., Schöllhorn, W.I.: Explaining the unique nature of individual gait patterns with deep learning 9(1), 2391. https://doi.org/10.1038/s41598-019-38748-8, https://www.nature.com/articles/s41598-019-38748-8

  12. Jain, A., Ross, A., Prabhakar, S.: An introduction to biometric recognition 14(1), 4–20. https://doi.org/10.1109/TCSVT.2003.818349, http://ieeexplore.ieee.org/document/1262027/

  13. Lampert, C.H., Nickisch, H., Harmeling, S.: Attribute-based classification for zero-shot visual object categorization 36(3), 453–465. https://doi.org/10.1109/TPAMI.2013.140, http://ieeexplore.ieee.org/document/6571196/

  14. Wang, L., Tan, T., Ning, H., Hu, W.: Silhouette analysis-based gait recognition for human identification 25(12), 1505–1518. https://doi.org/10.1109/TPAMI.2003.1251144, http://ieeexplore.ieee.org/document/1251144/

  15. Liu, L.-F., Jia, W., Zhu, Y.-H.: Survey of gait recognition. In: Huang, D.-S., Jo, K.-H., Lee, H.-H., Kang, H.-J., Bevilacqua, V. (eds.) ICIC 2009. LNCS (LNAI), vol. 5755, pp. 652–659. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04020-7_70

    Chapter  Google Scholar 

  16. Malekzadeh, M., Clegg, R.G., Cavallaro, A., Haddadi, H.: Mobile sensor data anonymization. In: Proceedings of the International Conference on Internet of Things Design and Implementation, pp. 49–58. ACM. https://doi.org/10.1145/3302505.3310068, https://dl.acm.org/doi/10.1145/3302505.3310068

  17. Mekruksavanich, S., Jitpattanakul, A.: Biometric user identification based on human activity recognition using wearable sensors: An experiment using deep learning models 10(3), 308. https://doi.org/10.3390/electronics10030308, https://www.mdpi.com/2079-9292/10/3/308

  18. Montavon, G., Binder, A., Lapuschkin, S., Samek, W., Müller, K.-R.: Layer-wise relevance propagation: An overview. In: Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., Müller, K.-R. (eds.) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. LNCS (LNAI), vol. 11700, pp. 193–209. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-28954-6_10

    Chapter  Google Scholar 

  19. Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Müller, K.R.: Explaining nonlinear classification decisions with deep Taylor decomposition 65, 211–222. https://doi.org/10.1016/j.patcog.2016.11.008, https://linkinghub.elsevier.com/retrieve/pii/S0031320316303582

  20. Moya Rueda, F., Grzeszick, R., Fink, G., Feldhorst, S., ten Hompel, M.: Convolutional neural networks for human activity recognition using body-worn sensors 5(2), 26. https://doi.org/10.3390/informatics5020026, http://www.mdpi.com/2227-9709/5/2/26

  21. Niemann, F., et al.: LARa: Creating a dataset for human activity recognition in logistics using semantic attributes 20(15), 4083. https://doi.org/10.3390/s20154083, https://www.mdpi.com/1424-8220/20/15/4083

  22. Reining, C., Rueda, F.M., Niemann, F., Fink, G.A., Hompel, M.T.: Annotation performance for multi-channel time series HAR dataset in logistics. In: 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 1–6. IEEE. https://doi.org/10.1109/PerComWorkshops48775.2020.9156170, https://ieeexplore.ieee.org/document/9156170/

  23. Reining, C., Schlangen, M., Hissmann, L., ten Hompel, M., Moya, F., Fink, G.A.: Attribute representation for human activity recognition of manual order picking activities. In: Proceedings of the 5th intl. Workshop on Sensor-based Activity Recognition and Interaction, pp. 1–10. ACM. https://doi.org/10.1145/3266157.3266214, https://dl.acm.org/doi/10.1145/3266157.3266214

  24. Reiss, A., Stricker, D.: Introducing a new benchmarked dataset for activity monitoring. In: 2012 16th International Symposium on Wearable Computers, pp. 108–109. IEEE. https://doi.org/10.1109/ISWC.2012.13, http://ieeexplore.ieee.org/document/6246152/

  25. Retsinas, G., Filntisis, P.P., Efthymiou, N., Theodosis, E., Zlatintsi, A., Maragos, P.: Person identification using deep convolutional neural networks on short-term signals from wearable sensors. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3657–3661. IEEE. https://doi.org/10.1109/ICASSP40776.2020.9053910, https://ieeexplore.ieee.org/document/9053910/

  26. Riaz, Q., Vögele, A., Krüger, B., Weber, A.: One small step for a man: Estimation of gender, age and height from recordings of one step by a single inertial sensor 15(12), 31999–32019. https://doi.org/10.3390/s151229907, http://www.mdpi.com/1424-8220/15/12/29907

  27. Rueda, F.M., Fink, G.A.: From human pose to on-body devices for human-activity recognition. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 10066–10073. IEEE. https://doi.org/10.1109/ICPR48806.2021.9412283, https://ieeexplore.ieee.org/document/9412283/

  28. Rueda, F.M., Fink, G.A.: Learning attribute representation for human activity recognition. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 523–528. IEEE. https://doi.org/10.1109/ICPR.2018.8545146, https://ieeexplore.ieee.org/document/8545146/

  29. Rusakov, E., Rothacker, L., Mo, H., Fink, G.A.: A probabilistic retrieval model for word spotting based on direct attribute prediction. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 38–43. IEEE. https://doi.org/10.1109/ICFHR-2018.2018.00016, https://ieeexplore.ieee.org/document/8563223/

  30. Shahid, S., Nandy, A., Mondal, S., Ahamad, M., Chakraborty, P., Nandi, G.C.: A study on human gait analysis. In: Proceedings of the Second International Conference on Computational Science, Engineering and Information Technology - CCSEIT 2012, pp. 358–364. ACM Press. https://doi.org/10.1145/2393216.2393277, http://dl.acm.org/citation.cfm?doid=2393216.2393277

  31. Singh, J.P., Jain, S., Arora, S., Singh, U.P.: Vision-based gait recognition: A survey 6, 70497–70527. https://doi.org/10.1109/ACCESS.2018.2879896, https://ieeexplore.ieee.org/document/8528404/

  32. Sudholt, S., Fink, G.A.: Attribute CNNs for word spotting in handwritten documents 21(3), 199–218. https://doi.org/10.1007/s10032-018-0295-0, http://link.springer.com/10.1007/s10032-018-0295-0

Download references

Acknowledgment

The work on this publication was supported by Deutsche Forschungsgemeinschaft (DFG) in the context of the project Fi799/10-2 “Transfer Learning for Human Activity Recognition in Logistics”.

Author information

Authors and Affiliations

  1. Chair of Material Handling and Warehousing, TU Dortmund University, Dortmund, Germany

    Nilah Ravi Nair & Christopher Reining

  2. Pattern Recognition in Embedded Systems Group, TU Dortmund University, Dortmund, Germany

    Fernando Moya Rueda & Gernot A. Fink

Authors
  1. Nilah Ravi Nair
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando Moya Rueda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christopher Reining
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gernot A. Fink
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nilah Ravi Nair .

Editor information

Editors and Affiliations

  1. York University, Toronto, ON, Canada

    Jean-Jacques Rousseau

  2. Ontario Tech University, Oshawa, ON, Canada

    Bill Kapralos

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nair, N.R., Moya Rueda, F., Reining, C., Fink, G.A. (2023). Multi-Channel Time-Series Person and Soft-Biometric Identification. In: Rousseau, JJ., Kapralos, B. (eds) Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges. ICPR 2022. Lecture Notes in Computer Science, vol 13643. Springer, Cham. https://doi.org/10.1007/978-3-031-37660-3_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-37660-3_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-37659-7

  • Online ISBN: 978-3-031-37660-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships