Skip to main content
SpringerLink
Log in

Chances of Interpretable Transfer Learning for Human Activity Recognition in Warehousing

  • Conference paper
  • First Online:
Computational Logistics (ICCL 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 13004))

Included in the following conference series:

Abstract

Human activity recognition evolves around classifying and analyzing workers’ actions quantitatively using convolutional neural networks on the time-series data provided by inertial measurement units and motion capture systems. However, this requires expensive training datasets since each warehouse scenario has slightly different settings and activities of interest. Here, transfer learning promises to shift the knowledge a deep learning method gained on existing reference data to new target data. We benchmark interpretable and non-interpretable transfer learning for human activity recognition on the LARa order-picking dataset with AndyLab and RealDisp as domain-related and domain-foreign reference datasets. We find that interpretable transfer learning via the recently proposed probabilistic rule stacking learner, which does not require any labeled data on the target dataset, is possible if the labels are sufficiently semantically related. The success depends on the proximity of the reference and target domains and labels. Non-interpretable transfer learning via fine-tuning can be applied even if there is a major domain-shift between the datasets and reduces the amount of labeled data required on the target dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agresti, A.: An Introduction to Categorical Data Analysis. John Wiley, Hoboken (2018)

    Google Scholar 

  2. Atzmon, Y., Chechik, G.: Probabilistic AND-OR attribute grouping for zero-shot learning. In: Conference on Uncertainty in Artificial Intelligence (2018)

    Google Scholar 

  3. Avsar, H., Altermann, E., Reining, C., Rueda, F.M., Fink, G.A., ten Hompel, M.: Benchmarking annotation procedures for multi-channel time series HAR dataset. In: 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, pp. 453–458 (2021)

    Google Scholar 

  4. Banos, O., Toth, M.A., Damas, M., Pomares, H., Rojas, I.: Dealing with the effects of sensor displacement in wearable activity recognition. Sensors 14(6), 9995–10023 (2014)

    Article  Google Scholar 

  5. Calzavara, M., Glock, C.H., Grosse, E.H., Persona, A., Sgarbossa, F.: Analysis of economic and ergonomic performance measures of different rack layouts in an order picking warehouse. Comput. Ind. Eng. 111, 527–536 (2017)

    Google Scholar 

  6. Chen, C., Jafari, R., Kehtarnavaz, N.: A survey of depth and inertial sensor fusion for human action recognition. Multimed. Tools Appl. 76(3), 4405–4425 (2017)

    Article  Google Scholar 

  7. Cheng, H.T., Sun, F.T., Griss, M., Davis, P., Li, J., You, D.: NuActiv: recognizing unseen new activities using semantic attribute-based learning. In: 11th Annual Conference on Mobile Systems, Applications, and Services, pp. 361–374 (2013)

    Google Scholar 

  8. Daduna, J.R.: Automated and autonomous driving in freight transport - opportunities and limitations. In: Lalla-Ruiz, E., Mes, M., Voß, S. (eds.) ICCL 2020. LNCS, vol. 12433, pp. 457–475. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59747-4_30

    Chapter  Google Scholar 

  9. Ding, N., Deng, J., Murphy, K.P., Neven, H.: probabilistic label relation graphs with Ising models. In: Proceedings of the 2015 IEEE International Conference on Computer Vision, pp. 1161–1169 (2015)

    Google Scholar 

  10. Feldhorst, S., Aniol, S., ten Hompel, M.: Human Activity Recognition in der Kommissionierung - Charakterisierung des Kommissionierprozesses als Ausgangsbasis für die Methodenentwicklung. Logistics J. 2016(10) (2016)

    Google Scholar 

  11. Grosse, E.H., Calzavara, M., Glock, C.H., Sgarbossa, F.: Incorporating human factors into decision support models for production and logistics: current state of research. IFAC-PapersOnLine 50(1), 6900–6905 (2017)

    Article  Google Scholar 

  12. Grosse, E.H., Glock, C.H., Neumann, W.P.: Human factors in order picking system design: a content analysis. IFAC-PapersOnLine 48(3), 320–325 (2015)

    Article  Google Scholar 

  13. Guo, L., Lei, Y., Xing, S., Yan, T., Li, N.: Deep convolutional transfer learning network: a new method for intelligent fault diagnosis of machines with unlabeled data. IEEE Trans. Industr. Electron. 66(9), 7316–7325 (2018)

    Article  Google Scholar 

  14. Huang, H., Pouls, M., Meyer, A., Pauly, M.: Travel time prediction using tree-based ensembles. In: Lalla-Ruiz, E., Mes, M., Voß, S. (eds.) ICCL 2020. LNCS, vol. 12433, pp. 412–427. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59747-4_27

    Chapter  Google Scholar 

  15. Kirchhof, M., Schmid, L., Reining, C., ten Hompel, M., Pauly, M.: pRSL: interpretable multi-label stacking by learning probabilistic rules. In: Uncertainty in Artificial Intelligence. PMLR (2021). (in press)

    Google Scholar 

  16. Kirchhof, M.: GitHub repository for this article (2021). https://github.com/mkirchhof/rslAppl

  17. de Koster, R., Le-Duc, T., Roodbergen, K.J.: Design and control of warehouse order picking: a literature review. Eur. J. Oper. Res. 182(2), 481–501 (2007)

    Article  Google Scholar 

  18. Krüger, A., Feldmann, F., Pauly, M., ten Hompel, M.: Einsatzmöglichkeiten maschineller Lernverfahren in einer dezentral organisierten Lagerverwaltung auf Basis intelligenter Behälter. Logistics J. Proc. 2020(12) (2020)

    Google Scholar 

  19. Kull, M., Perello Nieto, M., Kängsepp, M., Silva Filho, T., Song, H., Flach, P.: Beyond temperature scaling: obtaining well-calibrated multi-class probabilities with Dirichlet calibration. Adv. Neural. Inf. Process. Syst. 32, 12316–12326 (2019)

    Google Scholar 

  20. Liu, L., Zhou, T., Long, G., Jiang, J., Zhang, C.: Attribute propagation network for graph zero-shot learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34(04), pp. 4868–4875 (2020)

    Google Scholar 

  21. Maurice, P., et al.: Human movement and ergonomics: an industry-oriented dataset for collaborative robotics. Int. J. Robot. Res. 38(14), 1529–1537 (2019)

    Google Scholar 

  22. Rueda, F.M., Grzeszick, R., Fink, G.A., Feldhorst, S., Ten Hompel, M.: Convolutional neural networks for human activity recognition using body-worn sensors. Informatics 5(2), 26 (2018)

    Google Scholar 

  23. Niemann, F., et al.: LARa: creating a dataset for human activity recognition in logistics using semantic attributes. Sensors 20(15), 4083 (2020)

    Google Scholar 

  24. Ordóñez, F., Roggen, D.: Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 115 (2016)

    Google Scholar 

  25. Reining, C., Niemann, F., Rueda, F.M., Fink, G.A., ten Hompel, M.: Human activity recognition for production and logistics - a systematic literature review. Information 10(8), 245 (2019)

    Article  Google Scholar 

  26. Reining, C., Rueda, F.M., Niemann, F., Fink, G.A., ten Hompel, M.: Annotation performance for multi-channel time series HAR dataset in logistics. In: 2020 IEEE International Conference on Pervasive Computing and Communications Workshops, pp. 1–6 (2020)

    Google Scholar 

  27. Ribeiro, P.M.S., Matos, A.C., Santos, P.H., Cardoso, J.S.: Machine learning improvements to human motion tracking with IMUs. Sensors 20(21), 6383 (2020)

    Article  Google Scholar 

  28. Roggen, D., et al.: Collecting complex activity datasets in highly rich networked sensor environments. In: Seventh International Conference on Networked Sensing Systems (INSS), pp. 233–240 (2010)

    Google Scholar 

  29. Rueda, F.M., Fink, G.: From human pose to on-body devices for human-activity recognition. In: 26th International Conference on Pattern Recognition (ICPR), pp. 10066–10073 (2021)

    Google Scholar 

  30. Schaub, K., Caragnano, G., Britzke, B., Bruder, R.: The European assembly worksheet. Theor. Issues Ergon. Sci. 14(6), 616–639 (2013)

    Article  Google Scholar 

  31. Vicon: Full Body Modeling with Plug-in Gate (2017). https://docs.vicon.com/display/Nexus26/Full+body+modeling+with+Plug-in+Gait. Accessed 16 Mar 2021

  32. Xian, Y., Lampert, C.H., Schiele, B., Akata, Z.: Zero-shot learning - a comprehensive evaluation of the good, the bad and the ugly. IEEE Trans. Pattern Anal. Mach. Intell. 41(9), 2251–2265 (2018)

    Article  Google Scholar 

  33. Yordanova, K., et al.: Challenges in Annotation of useR Data for UbiquitOUs Systems: Results from the 1st ARDUOUS Workshop (2018). arXiv:1803.05843

  34. Zhang, A., et al.: Transfer learning with deep recurrent neural networks for remaining useful life estimation. Appl. Sci. 8(12), 2416 (2018)

    Article  Google Scholar 

  35. Zhang, A., Lipton, Z.C., Li, M., Smola, A.J.: Dive into Deep Learning (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, TU Dortmund University, 44221, Dortmund, Germany

    Michael Kirchhof, Lena Schmid & Markus Pauly

  2. Chair of Material Handling and Warehousing, TU Dortmund University, 44221, Dortmund, Germany

    Christopher Reining & Michael ten Hompel

Authors
  1. Michael Kirchhof
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lena Schmid
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christopher Reining
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael ten Hompel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Markus Pauly
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lena Schmid .

Editor information

Editors and Affiliations

  1. IEBIS, University of Twente, Enschede, Overijssel, The Netherlands

    Martijn Mes

  2. IEBIS, University of Twente, Enschede, Overijssel, The Netherlands

    Eduardo Lalla-Ruiz

  3. IWI-Institute of Information Systems, University of Hamburg, Hamburg, Germany

    Stefan Voß

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kirchhof, M., Schmid, L., Reining, C., Hompel, M.t., Pauly, M. (2021). Chances of Interpretable Transfer Learning for Human Activity Recognition in Warehousing. In: Mes, M., Lalla-Ruiz, E., Voß, S. (eds) Computational Logistics. ICCL 2021. Lecture Notes in Computer Science(), vol 13004. Springer, Cham. https://doi.org/10.1007/978-3-030-87672-2_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87672-2_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87671-5

  • Online ISBN: 978-3-030-87672-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation