Skip to main content
SpringerLink
Log in

Image-Based Out-of-Distribution-Detector Principles on Graph-Based Input Data in Human Action Recognition

  • Conference paper
  • First Online:
Pattern Recognition. ICPR International Workshops and Challenges (ICPR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12661))

Included in the following conference series:

Abstract

Living in a complex world like ours makes it unacceptable that a practical implementation of a machine learning system assumes a closed world. Therefore, it is necessary for such a learning-based system in a real world environment, to be aware of its own capabilities and limits and to be able to distinguish between confident and unconfident results of the inference, especially if the sample cannot be explained by the underlying distribution. This knowledge is particularly essential in safety-critical environments and tasks e.g. self-driving cars or medical applications. Towards this end, we transfer image-based Out-of-Distribution (OoD)-methods to graph-based data and show the applicability in action recognition.

The contribution of this work is (i) the examination of the portability of recent image-based OoD-detectors for graph-based input data, (ii) a Metric Learning-based approach to detect OoD-samples, and (iii) the introduction of a novel semi-synthetic action recognition dataset.

The evaluation shows that image-based OoD-methods can be applied to graph-based data. Additionally, there is a gap between the performance on intraclass and intradataset results. First methods as the examined baseline or ODIN provide reasonable results. More sophisticated network architectures – in contrast to their image-based application – were surpassed in the intradataset comparison and even lead to less classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Allan, J., Carbonell, J., Doddington, G., Yamron, J., Yang, Y.: Topic Detection and Tracking Pilot Study. Topic Detection and Tracking Workshop Report (2001)

    Google Scholar 

  2. Bayer, J., Münch, D., Arens, M.: Viewpoint Independency for Skeleton Based Human Action Recognition. Tech. rep, Fraunhofer IOSB, Ettlingen (2020)

    Google Scholar 

  3. Bhosale, S.V.: Holy Grail of Outlier Detection Technique: A Macro Level Take on the State of the Art. IJCSIT (2014)

    Google Scholar 

  4. Bishop, C.M.: Novelty detection and neural network validation. IEE Proc. Vision Image Signal Process. 141(4), 217–222 (1994)

    Article  Google Scholar 

  5. Blender Online Community: Blender - a 3D modelling and rendering package (2019)

    Google Scholar 

  6. Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: CVPR 2017-Janua, pp. 1302–1310 (2017)

    Google Scholar 

  7. Carnegie Mellon Graphics Lab: CMU Graphics Lab Motion Capture Database

    Google Scholar 

  8. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection. ACM Comput. Surv. 41(3), 1–58 (2009)

    Article  Google Scholar 

  9. Dasgupta, D., Forrest, S.: Novelty detection in time series data using ideas from immunology. In: The 8th International Conference on Intelligent Systems, pp. 82–87 (1999)

    Google Scholar 

  10. Devanne, M., Wannous, H., Berretti, S., Pala, P., Daoudi, M., Del Bimbo, A.: 3-D human action recognition by shape analysis of motion trajectories on riemannian manifold. IEEE Trans. Cybern. 45(7), 1340–1352 (2015)

    Article  Google Scholar 

  11. DeVries, T., Taylor, G.W.: Learning confidence for out-of-distribution detection in neural networks. arXiv preprint arXiv:1802.04865 (2018)

  12. Du, Y., Wang, W., Wang, L.: Hierarchical recurrent neural network for skeleton based action recognition. In: CVPR (2015)

    Google Scholar 

  13. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: Others: a density-based algorithm for discovering clusters in large spatial databases with noise. KDD 96, 226–231 (1996)

    Google Scholar 

  14. John, G.H.: Robust decision trees: removing outliers from databases. In: KDD, pp. 174–179 (1995)

    Google Scholar 

  15. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez-Gonzalez, P., Garcia-Rodriguez, J.: A survey on deep learning techniques for image and video semantic segmentation. Appl. Soft Comput. J. 70, 41–65 (2018)

    Google Scholar 

  16. Grubbs, F.E.: Procedures for detecting outlying observations in samples. Technometrics 11(1), 1–21 (1969)

    Article  Google Scholar 

  17. Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: CVPR, vol. 2, pp. 1735–1742. IEEE (2006)

    Google Scholar 

  18. Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: ICLR (2017)

    Google Scholar 

  19. Hilsenbeck, B., Münch, D., Kieritz, H., Hübner, W., Arens, M.: Hierarchical hough forests for view-independent action recognition. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 1911–1916 (2016)

    Google Scholar 

  20. Hinton, G., Vinyals, O., Dean, J.: Distilling the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531 (2015)

  21. Hodge, V.J., Austin, J.: A survey of outlier detection methodologies. Artif. Intell. Rev. 22(2), 85–126 (2004)

    Article  Google Scholar 

  22. Laurikkala, J., Juhola, M., Kentala, E.: Informal identification of outliers in medical data. In: IDAMAP, vol. 1, pp. 20–24 (2000)

    Google Scholar 

  23. Japkowicz, N., Myers, C., Gluck, M.: A novelty detection approach to classification. In: IJCAI, pp. 518–523, Montreal (1995)

    Google Scholar 

  24. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: CVPR (2019)

    Google Scholar 

  25. Kerola, T., Inoue, N., Shinoda, K.: Spectral graph skeletons for 3D action recognition. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 417–432. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16817-3_27

    Chapter  Google Scholar 

  26. Kliger, M., Fleishman, S.: Novelty Detection with GAN. arXiv preprint arXiv:1802.10560 (2018)

  27. Knox, E.M., Ng, R.T.: Algorithms for mining distance based outliers in large datasets. In: VLDB, pp. 392–403. Citeseer (1998)

    Google Scholar 

  28. Kong, Y., Fu, Y.: Human action recognition and prediction: a survey. arXiv preprint arXiv:1806.11230 (2018)

  29. Liang, S., Li, Y., Srikant, R.: Enhancing the reliability of out-of-distribution image detection in neural networks. In: ICLR (2018)

    Google Scholar 

  30. Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., Pietikäinen, M.: Deep learning for generic object detection: a survey. Int. J. Comput. Vision 128(2), 261–318 (2019). https://doi.org/10.1007/s11263-019-01247-4

    Article  Google Scholar 

  31. Masana, M., Ruiz, I., Serrat, J., van de Weijer, J., Lopez, A.M.: Metric learning for novelty and anomaly detection. In: BMVC (2018)

    Google Scholar 

  32. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29

    Chapter  Google Scholar 

  33. Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: CVPR, pp. 427–436 (2015)

    Google Scholar 

  34. Papadopoulos, G.T., Axenopoulos, A., Daras, P.: Real-time skeleton-tracking-based human action recognition using kinect data. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014. LNCS, vol. 8325, pp. 473–483. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-04114-8_40

    Chapter  Google Scholar 

  35. Papandreou, G., Zhu, T., Chen, L.-C., Gidaris, S., Tompson, J., Murphy, K.: PersonLab: person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 282–299. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_17

    Chapter  Google Scholar 

  36. Pidhorskyi, S., Adjeroh, D.A., Doretto, G.: Adversarial latent autoencoders. In: CVPR (2020)

    Google Scholar 

  37. Poppe, R.: A survey on vision-based human action recognition. Image Vis. Comput. 28(6), 976–990 (2010)

    Article  Google Scholar 

  38. Seheult, A.H., Green, P.J., Rousseeuw, P.J., Leroy, A.M.: Robust regression and outlier detection. J. Royal Stat. Soc. Ser. A (Stat. Soc.) 152(1), 133 (1989)

    Google Scholar 

  39. Shahroudy, A., Liu, J., Ng, T.T., Wang, G.: Ntu rgb+d: a large scale dataset for 3d human activity analysis. In: CVPR (2016)

    Google Scholar 

  40. Si, C., Jing, Y., Wang, W., Wang, L., Tan, T.: Skeleton-based action recognition with spatial reasoning and temporal stack learning. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11205, pp. 106–121. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01246-5_7

    Chapter  Google Scholar 

  41. Szegedy, C., et al.: Intriguing properties of neural networks. In: ICLR (2014)

    Google Scholar 

  42. Tax, D., Ypma, A., Duin, R.: Support vector data description applied to machine vibration analysis. In: Proceedings of 5th Annual Conference of the Advanced School for Computing and Imaging, pp. 15–23 (1999)

    Google Scholar 

  43. The MakeHuman team: MakeHuman. www.makehumancommunity.org. Accessed 15 Nov 2020

  44. Turaga, P., Chellappa, R., Subrahmanian, V.S., Udrea, O.: Machine recognition of human activities: A survey. TCSVT 18(11), 1473–1488 (2008)

    Google Scholar 

  45. Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: AAAI, pp. 7444–7452 (2018)

    Google Scholar 

  46. Zhao, M., et al.: Through-wall human pose estimation using radio signals. In: CVPR (2018)

    Google Scholar 

  47. Zimek, A., Filzmoser, P.: There and back again: outlier detection between statistical reasoning and data mining algorithms (2018)

    Google Scholar 

Download references

Acknowledgements

This work was developed in Fraunhofer Cluster of Excellence “Cognitive Internet Technologies”.

Portions of the research in this paper used the NTU-RGB+D Action Recognition Dataset made available by the ROSE Lab at the Nanyang Technological University, Singapore.

Author information

Authors and Affiliations

  1. Fraunhofer Center for Machine Learning, Karlsruhe, Germany

    Jens Bayer, David Münch & Michael Arens

  2. Fraunhofer IOSB, Gutleuthausstraße 1, 76275, Ettlingen, Germany

    Jens Bayer, David Münch & Michael Arens

Authors
  1. Jens Bayer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Münch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Arens
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jens Bayer .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Alberto Del Bimbo

  2. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Rita Cucchiara

  3. Department of Computer Science, Boston University, Boston, MA, USA

    Stan Sclaroff

  4. Dipartimento di Matematica e Informatica, University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Cloud & AI, JD.COM, Beijing, China

    Tao Mei

  6. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Marco Bertini

  7. Computational Sciences Department, National Institute of Astrophysics, Optics and Electronics (INAOE), Tonantzintla, Puebla, Mexico

    Hugo Jair Escalante

  8. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Roberto Vezzani

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bayer, J., Münch, D., Arens, M. (2021). Image-Based Out-of-Distribution-Detector Principles on Graph-Based Input Data in Human Action Recognition. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12661. Springer, Cham. https://doi.org/10.1007/978-3-030-68763-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68763-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68762-5

  • Online ISBN: 978-3-030-68763-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation