Introduction of a time series machine learning methodology for the application in a production system

https://doi.org/10.1016/j.aei.2020.101197Get rights and content

Abstract

Machine learning methods are considered a promising approach for improving operations and processes in manufacturing. However, the application of machine learning often requires the expertise of a data scientist combined with thorough knowledge of the manufacturing processes. Small and medium-sized companies that specialize in certain high value-added, variant rich production processes often lack an in-house data scientist and therefore miss out on generating a deeper data-driven insight from their production data streams. This paper proposes a three-step machine learning methodology to empower process experts with limited knowledge in machine learning: 1) data exploration through clustering, 2) representation of the production systems behaviour through specially structured neural networks and 3) querying this representation through evolutionary algorithms to achieve decision support through online optimization or scenario simulation. The chosen algorithms focus on parameter-light, well-established, general use algorithms in order to lower knowledge requirements for their application.

Section snippets

Introduction, motivation

The manufacturing sector is experiencing an unprecedented increase in data availability. This data has a variety of formats, semantics, and quality levels as it comprises e.g. sensor data from a production line, machine tool parameters from production assets, material datasets from suppliers and overhead data [83]. The data sources are utilized in order to address increasing requirements for highly flexible production environments working in a decentralized interconnected manner [19]. Smart

Related work

The concept of ML is known for several decades and its general understanding still holds. It is a set of methods for programming computers, to learn to solve problems or make predictions based on training data rather than a predefined procedure [69]. ML methods deliver a promising approach for addressing the key future challenges in the manufacturing sector especially in the context of industrial big data.

When ML is applied to industrial TS, the most popular approaches are unsupervised ML [79]

Concept

The proposed methodology comprises three consecutive steps, as depicted in the graphical abstract. The first step is exploratory and concerned with clustering the TS. Structure is found by grouping reveal reoccurring similar patterns together and interpreting the groups in a meaningful way (e.g. a machine operation state). In the second step this structure is utilized, training a NN as a model for the system’s behaviour to obtain a learned representation. The structure of the NN is modularized

Proof of concept

This use case was chosen because it has defined input and output TS and a known logical dependence of the production equipment components (e.g. plant topology). The process is a test setup of an industrial winding process as depicted in Fig. 2. The data is publicly available and intended for testing system identification approaches. In the proof of concept use case a plastic web is unwound from a first reel, tracked over a traction reel and finally rewound on a third reel. Tachometers measure

Conclusion and outlook

This this work a ML methodology was introduced which combines a clustering of TS, representing them with MNNs and using this representation for optimization purposes like parameter tuning, in order to gain more benefits combined than the algorithms alone

From the proof of concept with a real-world production dataset, it is evident that finding reoccurring motifs in TS data first and using them in the training of a NN for regression leads to advantages in accuracy. Furthermore, utilizing the

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

The authors would like to acknowledge and express their sincere gratitude to the Austrian Research Promotion Agency (FFG) through grant project number 865898 and the Institute of Engineering Design and Product Development, Department Mechanical Engineering Informatics and Virtual Product Development for granting the research activities as part for a pre-doc position.

References (85)

  • P.J. Rousseeuw

    Silhouettes - a graphical aid to the interpretation and validation of cluster-analysis

    J. Comput. Appl. Math.

    (1987)
  • T. Warren Liao

    Clustering of time series data—a survey

    Pattern Recogn.

    (2005)
  • S.Y. Yim

    Using process topology in plant-wide control loop performance assessment

    Comput. Chem. Eng.

    (2006)
  • B. Alkan

    Complexity in manufacturing systems and its measures: a literature review

    Eur. J. Ind. Eng.

    (2018)
  • M. Amer et al.

    A review of modularization techniques in artificial neural networks

    Artif. Intell. Rev.

    (2019)
  • R. Anand

    Efficient classification for multiclass problems using modular neural networks

    IEEE Trans. Neural Networks

    (1995)
  • M. Ankerst

    OPTICS: ordering points to identify the clustering structure. ACM Sigmod record

    (1999)
  • G. Auda et al.

    Modular neural networks: a survey

    Int. J. Neural Syst.

    (1999)
  • T. Bäck, F. Hoffmeister, Extended selection mechanisms in genetic algorithms,...
  • T. Bäck et al.

    An overview of evolutionary algorithms for parameter optimization

    Evol. Comput.

    (1993)
  • T. Bastogne, Identification des systèmes multivariables par les méthodes des sous-espaces. Application à un système...
  • T. Bastogne

    Application of subspace methods to the identification of a winding process

    1997 European Control Conference (ECC), IEEE

    (1997)
  • D.J. Berndt, J. Clifford, Using Dynamic Time Warping to Find Patterns in Time Series. KDD workshop, Seattle, WA,...
  • A. Bey-Temsamani

    A practical approach to combine data mining and prognostics for improved predictive maintenance

    Data Min. Case Stud.

    (2009)
  • A. Blum et al.

    Training a 3-node neural network is NP-complete

    Adv. Neural Inf. Process. Syst.

    (1989)
  • M. Brettel

    How virtualization, decentralization and network building change the manufacturing landscape: an Industry 4.0 Perspective

    Int. J. Mech. Ind. Sci. Eng.

    (2014)
  • T. Caliński et al.

    A dendrite method for cluster analysis

    Commun. Stat.-Theory Methods

    (1974)
  • Y. Carson et al.

    Simulation optimization: methods and applications

    Proceedings of the 29th conference on Winter simulation

    (1997)
  • B.T. Chen

    Smart factory of industry 4.0: key technologies, application case, and challenges

    IEEE Access

    (2018)
  • R. Cheng et al.

    Genetic Algorithms and Engineering Design

    (1997)
  • B. Chiu

    Probabilistic discovery of time series motifs

    Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining

    (2003)
  • B. Dasgupta

    On the complexity of training neural networks with continuous activation functions

    IEEE Trans. Neural Netw.

    (1995)
  • D.L. Davies et al.

    A cluster separation measure

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1979)
  • B.L.R. De Moor, DaISy: Database for the Identification of Systems. [Used dataset: Data from a test setup of an...
  • E. Diaconescu

    The use of NARX neural networks to predict chaotic time series

    Wseas Trans. Comput. Res.

    (2008)
  • U. Dombrowski, et al., Concept for a cyber physical assembly system, in: Assembly and Manufacturing (ISAM), 2013 IEEE...
  • R. Erkki, P. Johnsson, Quality Data Management in the Next Industrial Revolution: A Study of Prerequisites for Industry...
  • P. Esling et al.

    Time-series data mining

    ACM Comput. Surv. (CSUR)

    (2012)
  • R.J. Frank

    Time series prediction and neural networks

    J. Intell. Rob. Syst.

    (2001)
  • M.C. Fu

    Optimization via simulation: a review

    Ann. Oper. Res.

    (1994)
  • J.C.B. Gamboa, Deep learning for time-series analysis. arXiv preprint arXiv:1701.01887,...
  • D. Gerhard

    Product lifecycle management challenges of CPPS

  • Cited by (9)

    • Hybrid combinatorial remanufacturing strategy for medical equipment in the pandemic

      2022, Computers and Industrial Engineering
      Citation Excerpt :

      Neural networks are suitable to solve emergency production problems with dynamic environments, heterogeneous data, and implicit interactions. Hennig et al. (2021) use neural networks to describe production actions, in which the recurring motifs are clustered first and then used for neural network regression training. Wu et al. (2020) propose a real-time parallel production scheduling algorithm based on the end-to-end neural network for the large-scale demand for masks during the COVID-19 pandemic.

    • Time series clustering via matrix profile and community detection

      2022, Advanced Engineering Informatics
      Citation Excerpt :

      This amount of time series data has provided the opportunity to gain insight into these domains through data mining technology. Time series clustering [11–13] is a valuable data mining tool with an aim to group similar sequences together into clusters, such that the distances between the sequences in a cluster are the smallest, and the distances between the sequences in different clusters are the largest. Time series clustering algorithms have been used extensively for the discovery of interesting patterns in time series datasets [14].

    View all citing articles on Scopus
    View full text