Skip to main content
Springer Nature Link
Log in

A Benchmark for Real-Time Anomaly Detection Algorithms Applied in Industry 4.0

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Data Science (LOD 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13810))

  • 1206 Accesses

Abstract

Industry 4.0 describes flexibly combinable production machines enabling efficient fulfillment of individual requirements. Timely and automated anomaly recognition by means of machine self-diagnosis might support efficiency. Various algorithms have been developed in recent years to detect anomalies in data streams. Due to their diverse functionality, the application of different real-time anomaly detection algorithms to the same data stream may lead to different results. Existing algorithms as well as mechanisms for their evaluation and selection are context-independent and not suited to industry 4.0 settings. In this research paper, an industry 4.0 specific benchmark for real-time anomaly detection algorithms is developed on the basis of six design principles in the categories timeliness, threshold setting and qualitative classification. Given context-specific input parameters, the benchmark ranks algorithms according to their suitability for real-time anomaly detection in production datasets. The application of the benchmark is demonstrated and evaluated on the basis of two case studies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Adams, E.P., MacKay, D.J.C. Bayesian Online Changepoint Detection (2007)

    Google Scholar 

  • Ahmad, S., Lavin, A., Purdy, S., et al.: Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262, 134–147 (2017)

    Article  Google Scholar 

  • Bischl, B., et al.: ASlib: a benchmark library for algorithm selection. Artif. Intell. 237, 41–58 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Brazdil, P.B., Soares, C.: A comparison of ranking methods for classification algorithm selection. In: LópezdeMántaras, R., Plaza, E. (eds.) ECML 2000. LNCS (LNAI), vol. 1810, pp. 63–75. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45164-1_8

    Chapter  Google Scholar 

  • Bueno, A., Godinho Filho, M., Frank, A.G.: ‘Smart production planning and control in the Industry 4.0 context: a systematic literature review, Comput. Industrial Eng. 149, 106774 (2020)

    Google Scholar 

  • Burnaev, E., Ishimtsev, V.: Conformalized density- and distance-based anomaly detection in time-series data (2016). https://arxiv.org/abs/1608.04585. Accessed 16 Nov 2021

  • Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection. ACM Comput. Surv. 41(3), 1–58 (2009)

    Article  Google Scholar 

  • Cheng, C.-Y., Pourhejazy, P., Ying, K.-C., Liao, Y.-H.: New benchmark algorithms for No-wait flowshop group scheduling problem with sequence-dependent setup times. Appl. Soft Comput. 111, 107705 (2021)

    Google Scholar 

  • Cohen, Y., Singer, G.: A smart process controller framework for Industry 4.0 settings. J. Intell. Manuf. 32(7), 1975–1995 (2021). https://doi.org/10.1007/s10845-021-01748-5

    Article  Google Scholar 

  • Emmott, A.F., Das, S., Dietterich, T., Fern, A., Wong, W.-K.: Systematic construction of anomaly detection benchmarks from real data. In: Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description - ODD 2013. Chicago, Illinois, New York, New York, USA, 16–21 (2013)

    Google Scholar 

  • Hevner, A.R., March, S., Park, J., Ram, S.: Design science in information systems research. MIS Q. 28(1), 75 (2004)

    Article  Google Scholar 

  • Ho, Y.C., Pepyne, D.L.: Simple explanation of the no-free-lunch theorem and its implications. J. Optim. Theory Appl. 115(3), 549–570 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  • Kagermann, H., Wahlster, W., Helbig, J.: Umsetzungsempfehlungen für das Zukunftsprojekt Industrie 4.0: Abschlussbericht des Arbeitskreises Industrie 4.0 (2013). https://www.acatech.de/wp-content/uploads/2018/03/Abschlussbericht_Industrie4.0_barrierefrei.pdf. Accessed 16 Nov 2021

  • Kerschke, P., Kotthoff, L., Bossek, J., et al.: Leveraging TSP solver complementarity through machine learning. Evol. Comput. 26(4), 597–620 (2018)

    Article  Google Scholar 

  • Lavin, A., Ahmad, S.: Evaluating Real-Time Anomaly Detection Algorithms - The Numenta Anomaly Benchmark (2015). https://arxiv.org/abs/1510.03336. Accessed 16 Nov 2021

  • Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, 3rd edn. Springer, New York (2005). https://doi.org/10.1007/0-387-27605-X

    Book  MATH  Google Scholar 

  • Mayring, P.: Qualitative content analysis: theoretical foundation, basic procedures and software solution. SSOAR, Klagenfurt (2014)

    Google Scholar 

  • Numenta Anomaly Benchmark (2020). Numenta Anomaly Benchmark. URL: https://github.com/numenta/NAB (visited on November 16, 2021)

  • Pavelski, L., Delgado, M., Kessaci, M.-E.: Meta-Learning for Optimization: A Case Study on the Flowshop Problem Using Decision Trees. IEEE Congress on Evolutionary Computation, pp. 1–8, Rio de Janeiro, Brazil (2018)

    Google Scholar 

  • Peffers, K., Tuunanen, T., Rothenberger, M.A., Chatterjee, S.: A design science research methodology for information systems research. J. Manag. Inf. Syst. 24(3), 45–77 (2007)

    Article  Google Scholar 

  • Schütze, A., Helwig, N., Schneider, T.: Sensors 4.0 – smart sensors and measurement technology enable Industry 4.0. J. Sens. Sens. Syst. 7(1), 359–371 (2018)

    Article  Google Scholar 

  • Schneider, M., Ertel, W., Ramos, F.: Expected similarity estimation for large-scale batch and streaming anomaly detection. Mach. Learn. 105(3), 305–333 (2016). https://doi.org/10.1007/s10994-016-5567-7

    Article  MathSciNet  MATH  Google Scholar 

  • Singh, N., Olinsky, C.: Demystifying Numenta anomaly benchmark. In: 2017 International Joint Conference on Neural Networks (IJCNN), 1570–1577), IEEE (2017)

    Google Scholar 

  • Stahmann, P., Rieger, B.: Requirements Identification for Real-Time Anomaly Detection in Industrie 4.0 Machine Groups: A Structured Literature Review’, Proceedings of the 54th Hawaii International Conference on System Sciences (2021)

    Google Scholar 

  • Stahmann, P., Rieger, B.: Towards design principles for a real-time anomaly detection algorithm benchmark suited to Industrie 4.0 streaming data, Proceedings of the 55th Hawaii International Conference of System Sciences (2022a)

    Google Scholar 

  • Stahmann, P., Oodes, J., Rieger, B.: Improving machine self-diagnosis with an instance-based selector for real-time anomaly detection algorithms. In: 8th International Conference on Decision Support System Technology 2022b. Decision Support addressing modern Industry, Business and Societal needs, Lecture Notes in Business Information Processing (LNBIP) (2022b)

    Google Scholar 

  • Tukey, J.W.: Exploratory Data Analysis. Addison-Wesley Series in Behavioral Science Quantitative Methods. Addison-Wesley, Reading, Mass. (1977)

    Google Scholar 

  • Venable, J., PriesHeje, J., Baskerville, R.: FEDS: a framework for evaluation in design science research. Eur. J. Inf. Syst. 25(1), 77–89 (2016)

    Article  Google Scholar 

  • Vom Brocke, J., Simons, A., Riemer, K., Niehaves, B., Plattfaut, R., Cleven, A.: Standing on the shoulders of giants: challenges and recommendations of literature search in information systems research. Commun. Assoc. Inf. Syst. 37 (2015)

    Google Scholar 

  • Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Osnabrueck, Osnabrueck, Germany

    Philip Stahmann & Bodo Rieger

Authors
  1. Philip Stahmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bodo Rieger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philip Stahmann .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Italy

    Giuseppe Nicosia

  2. University of Reading, Reading, UK

    Varun Ojha

  3. University of Oxford, Oxford, UK

    Emanuele La Malfa

  4. University of Cambridge, Cambridge, UK

    Gabriele La Malfa

  5. University of Florida, Gainesville, FL, USA

    Panos Pardalos

  6. Free University of Bozen-Bolzano, Bolzano, Italy

    Giuseppe Di Fatta

  7. University of Catania, Catania, Italy

    Giovanni Giuffrida

  8. Dana-Farber Cancer Institute, Boston, MA, USA

    Renato Umeton

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Stahmann, P., Rieger, B. (2023). A Benchmark for Real-Time Anomaly Detection Algorithms Applied in Industry 4.0. In: Nicosia, G., et al. Machine Learning, Optimization, and Data Science. LOD 2022. Lecture Notes in Computer Science, vol 13810. Springer, Cham. https://doi.org/10.1007/978-3-031-25599-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25599-1_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25598-4

  • Online ISBN: 978-3-031-25599-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation