Introduction of a time series machine learning methodology for the application in a production system
Graphical abstract
Section snippets
Introduction, motivation
The manufacturing sector is experiencing an unprecedented increase in data availability. This data has a variety of formats, semantics, and quality levels as it comprises e.g. sensor data from a production line, machine tool parameters from production assets, material datasets from suppliers and overhead data [83]. The data sources are utilized in order to address increasing requirements for highly flexible production environments working in a decentralized interconnected manner [19]. Smart
Related work
The concept of ML is known for several decades and its general understanding still holds. It is a set of methods for programming computers, to learn to solve problems or make predictions based on training data rather than a predefined procedure [69]. ML methods deliver a promising approach for addressing the key future challenges in the manufacturing sector especially in the context of industrial big data.
When ML is applied to industrial TS, the most popular approaches are unsupervised ML [79]
Concept
The proposed methodology comprises three consecutive steps, as depicted in the graphical abstract. The first step is exploratory and concerned with clustering the TS. Structure is found by grouping reveal reoccurring similar patterns together and interpreting the groups in a meaningful way (e.g. a machine operation state). In the second step this structure is utilized, training a NN as a model for the system’s behaviour to obtain a learned representation. The structure of the NN is modularized
Proof of concept
This use case was chosen because it has defined input and output TS and a known logical dependence of the production equipment components (e.g. plant topology). The process is a test setup of an industrial winding process as depicted in Fig. 2. The data is publicly available and intended for testing system identification approaches. In the proof of concept use case a plastic web is unwound from a first reel, tracked over a traction reel and finally rewound on a third reel. Tachometers measure
Conclusion and outlook
This this work a ML methodology was introduced which combines a clustering of TS, representing them with MNNs and using this representation for optimization purposes like parameter tuning, in order to gain more benefits combined than the algorithms alone
From the proof of concept with a real-world production dataset, it is evident that finding reoccurring motifs in TS data first and using them in the training of a NN for regression leads to advantages in accuracy. Furthermore, utilizing the
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgement
The authors would like to acknowledge and express their sincere gratitude to the Austrian Research Promotion Agency (FFG) through grant project number 865898 and the Institute of Engineering Design and Product Development, Department Mechanical Engineering Informatics and Virtual Product Development for granting the research activities as part for a pre-doc position.
References (85)
Design of experiments on neural network's training for nonlinear time series forecasting
Neurocomputing
(2009)- et al.
Artificial neural networks: fundamentals, computing, design, and application
J. Microbiol. Methods
(2000) Forecasting the behavior of multivariate time-series using neural networks
Neural Netw.
(1992)- et al.
Identifying the source of variance shifts in the multivariate process using neural networks and support vector machines
Expert Syst. Appl.
(2008) Comparison among five evolutionary-based optimization algorithms
Adv. Eng. Inf.
(2005)An explanation of ordinal optimization: Soft computing for hard problems
Inform. Sci.
(1999)Comparison of a physical and a data-driven model of a Packed Bed Regenerator for industrial applications
J. Storage Mater.
(2019)- et al.
Long-term time series prediction with the NARX network: an empirical evaluation
Neurocomputing
(2008) Cyber-physical production systems: Roots, expectations and R&D challenges
Procedia CIRP
(2014)Evolutionary computing in manufacturing industry: an overview of recent applications
Appl. Soft Comput.
(2005)
Silhouettes - a graphical aid to the interpretation and validation of cluster-analysis
J. Comput. Appl. Math.
Clustering of time series data—a survey
Pattern Recogn.
Using process topology in plant-wide control loop performance assessment
Comput. Chem. Eng.
Complexity in manufacturing systems and its measures: a literature review
Eur. J. Ind. Eng.
A review of modularization techniques in artificial neural networks
Artif. Intell. Rev.
Efficient classification for multiclass problems using modular neural networks
IEEE Trans. Neural Networks
OPTICS: ordering points to identify the clustering structure. ACM Sigmod record
Modular neural networks: a survey
Int. J. Neural Syst.
An overview of evolutionary algorithms for parameter optimization
Evol. Comput.
Application of subspace methods to the identification of a winding process
1997 European Control Conference (ECC), IEEE
A practical approach to combine data mining and prognostics for improved predictive maintenance
Data Min. Case Stud.
Training a 3-node neural network is NP-complete
Adv. Neural Inf. Process. Syst.
How virtualization, decentralization and network building change the manufacturing landscape: an Industry 4.0 Perspective
Int. J. Mech. Ind. Sci. Eng.
A dendrite method for cluster analysis
Commun. Stat.-Theory Methods
Simulation optimization: methods and applications
Proceedings of the 29th conference on Winter simulation
Smart factory of industry 4.0: key technologies, application case, and challenges
IEEE Access
Genetic Algorithms and Engineering Design
Probabilistic discovery of time series motifs
Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining
On the complexity of training neural networks with continuous activation functions
IEEE Trans. Neural Netw.
A cluster separation measure
IEEE Trans. Pattern Anal. Mach. Intell.
The use of NARX neural networks to predict chaotic time series
Wseas Trans. Comput. Res.
Time-series data mining
ACM Comput. Surv. (CSUR)
Time series prediction and neural networks
J. Intell. Rob. Syst.
Optimization via simulation: a review
Ann. Oper. Res.
Product lifecycle management challenges of CPPS
Cited by (9)
Dynamic production bottleneck prediction using a data-driven method in discrete manufacturing system
2023, Advanced Engineering InformaticsHybrid combinatorial remanufacturing strategy for medical equipment in the pandemic
2022, Computers and Industrial EngineeringCitation Excerpt :Neural networks are suitable to solve emergency production problems with dynamic environments, heterogeneous data, and implicit interactions. Hennig et al. (2021) use neural networks to describe production actions, in which the recurring motifs are clustered first and then used for neural network regression training. Wu et al. (2020) propose a real-time parallel production scheduling algorithm based on the end-to-end neural network for the large-scale demand for masks during the COVID-19 pandemic.
Time series clustering via matrix profile and community detection
2022, Advanced Engineering InformaticsCitation Excerpt :This amount of time series data has provided the opportunity to gain insight into these domains through data mining technology. Time series clustering [11–13] is a valuable data mining tool with an aim to group similar sequences together into clusters, such that the distances between the sequences in a cluster are the smallest, and the distances between the sequences in different clusters are the largest. Time series clustering algorithms have been used extensively for the discovery of interesting patterns in time series datasets [14].
Investigation of artificial intelligence in SMEs: a systematic review of the state of the art and the main implementation challenges
2024, Management Review QuarterlyTime Series Prediction Method of Industrial Process With Limited Data Based on Transfer Learning
2023, IEEE Transactions on Industrial Informatics