KAiPP: An interaction recommendation approach for knowledge aided intelligent process planning with reinforcement learning
Introduction
Rapid advances in new generation information technologies, such as big data analytics, deep learning, and knowledge engineering, have been the main driving force for the revolution of product lifecycle [1], like design, manufacturing, and maintenance, so has it been for process planning [2], [3]. Process planning is an important link between design and manufacturing [4]. It determines, in detail, the process that will transform raw material into the form specified in design drawings. Process planning involves a series of decision-making activities that determine the production cost, quality, and time to market and affect all manufacturing activities. However, process decisions are more of an art than a science [5], which still depend on the technologists’ experience, skill, and intuition, resulting in labor intensive and time consuming. Besides, each of process decisions involves many intangibles, such as manufacturing quality, time, cost, etc., that need to be traded off. In this context, knowledge may help technologists understand intangibles, in order to develop good judgements to make decisions about these intangibles. Hence, effective reuse of historical knowledge in process planning of a new part could help technologists make scientific sound process decisions for that part, which brings improvements in production cost, quality, and time to market.
Effective knowledge reuse has been regarded as one of the most important technical roadmaps driving traditional human experience-based process planning all the way to intelligent process planning, which has attracted more and more attentions from both academia and industry. Nowadays, many researches have contributed to this field with the aim of retrieving the knowledge of similar parts for the process planning of a new part. The current researches could indeed recommend a group of knowledge individually for a specific process planning stage based on the current explicit needs of technologists captured by the keyword-based [6], model recognition-based [7] or context/semantics-based [8] approaches. The above approaches provide an insight into research and development of a knowledge reuse system in process planning, while still remaining several unaddressed issues that affect the effectiveness and efficiency of the system. Firstly, the above approaches regard the knowledge recommendation as a static process with a fixed strategy, which could hardly capture the dynamic preference of technologists as it is tacit and embedded in the interaction history of technologists with the system. In addition, almost no research has ever considered the mutual influence of the sequence of process decisions for knowledge reuse. In fact, the sequence of process decisions affects each other, which may significantly influence the effectiveness of the process knowledge recommendation. For example, if the first decision is to select a machine tool, then its power, spindle torque moment, and available speed range act as constraints for selecting cutting parameters. If another machine is selected, another set of constraints would arise.
With the above observations, this work motivates to develop a more effective and efficient knowledge recommendation system by not only considering current explicit needs of technologists expressed in query input, but also considering the tacit and dynamic preference of technologists. To this end, we take in-depth integration of deep learning and reinforcement learning in process planning and propose a novel dynamic interaction recommendation approach for knowledge aided intelligent process planning (KAiPP) towards Industry 4.0. This work firstly formalizes KAiPP as a sequential interaction knowledge recommendation scenario that considers both the current explicit needs and tacit preferences of technologists, to which a Markov decision process (MDP) is utilized to model the KAiPP scenario. Then, the gated recurrent unit (GRU) neural network and translating embedding (TransE) method are integrated with reinforcement learning to learn a KAiPP agent to better interact with the KAiPP environment for effective interaction knowledge recommendation. Here, the technologists’ preference-task information (TP-TI) model and dynamic knowledge sub-graph (DKS) are elaborated as a KAiPP environment to capture the preference of the technologist and mutual influence of process decision-making activities. Finally, experimental results demonstrate the effectiveness of the proposed approach.
The remainder of the paper is organized as follows. Section 2 summarizes the state-of-the-art researches related to this work. Section 3 formalizes the system framework of KAiPP. Section 4 introduces the key methodologies of KAiPP based on the framework. Section 5 presents the experimental results to demonstrate the effectiveness of the approach. The conclusions and future works are found in Section 6.
Section snippets
Related work
This section reviews the state-of-the-art researches related to this work, including knowledge-based process planning and reinforcement learning. Then, the current research gap and motivation of this work are highlighted.
System formalization of KAiPP
This section summarizes the issues influencing the performance of current approaches for knowledge reuse in process planning, on which a new process planning scenario, namely KAiPP is formalized. On that basis, the research problem of KAiPP is defined and the system framework of KAiPP is designed.
Key methodologies of KAiPP
Based on the system framework of KAiPP, this section introduces three key enabling technologies of KAiPP, including RL-oriented MDP formalization, TP-TI learning with GRU, and DKS representation with TransE.
Experiments
This section firstly analyses the experimental settings and results of the proposed approach. Then, a KAiPP prototype is implemented and its application examples show the practicability of the approach.
Conclusion and future work
This work takes the in-depth integration of deep learning and reinforcement learning in process planning and proposes a novel dynamic interaction recommendation approach for KAiPP towards Industry 4.0. Based on the experimental results presented in this work, the following contributions of the work could be concluded. Firstly, this work formalizes a new framework in process planning named KAiPP, which models the process knowledge recommendation as a dynamic process with MDP rather than a static
CRediT authorship contribution statement
Chao Zhang: Conceptualization, Methodology, Software, Investigation, Writing – original draft. Guanghui Zhou: Resources, Writing – review & editing, Supervision. Jingjing Li: Investigation, Writing – original draft. Tianyu Qin: Validation, Software, Data curation. Kai Ding: Writing – review & editing. Fengtian Chang: Writing – review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by the National Key Research and Development Program of China [grant number 2021YFB3301400], the National Natural Science Foundation of China [grant number 52105530], the China Postdoctoral Science Foundation [grant number 2021M692556], and the Key Research and Development Program of Shaanxi Province of China [under Grant 2022GY-261].
References (40)
- et al.
Implications of cyber-physical production systems on integrated process planning and scheduling
Procedia. Manuf.
(2019) - et al.
Hybrid manufacturing process planning for arbitrary part and tool shapes
Comput. Aided Des.
(2022) - et al.
Deep learning enabled cutting tool selection for special-shaped machining features of complex products
Adv. Eng. Softw.
(2019) - et al.
NC process reuse oriented effective subpart retrieval approach of 3D CAD models
Comput. Ind.
(2017) - et al.
A view-based 3D CAD model reuse framework enabling product lifecycle reuse
Adv. Eng. Softw.
(2019) - et al.
e-CAPP: A distributed collaborative system for internet-based process planning
J. Manuf. Syst.
(2017) - et al.
Framework for manufacturing-tasks semantic modelling and manufacturing-resource recommendation for digital twin shop-floor
J. Manuf. Syst.
(2021) An ontology-oriented knowledge methodology for process planning in additive layer manufacturing
Robot. Cim-Int. Manuf.
(2018)- et al.
A geometric reasoning approach to hierarchical representation for B-rep model retrieval
Comput. Aided. Des.
(2015) - et al.
An efficient assembly retrieval method based on Hausdorff distance
Robot. Cim-Int. Manuf.
(2018)
Deep learning-enabled intelligent process planning for digital twin manufacturing cell
Knowl. Based Syst.
Automatic question generation with various difficulty levels based on knowledge ontology using a query template
Knowl. Based Syst.
Topology based 2D engineering drawing and 3D model matching for process plant
Graph Models
An ontology-based semantic retrieval approach for heterogeneous 3D CAD models
Adv. Eng. Inform.
Deep learning, reinforcement learning, and world models
Neural Netw.
A Markov decision process approach for managing medical drone deliveries
Expert Syst. Appl.
Randomized filtering and Bellman equation in Wasserstein space for partial observation control problem
Stochastic Process. Appl.
Humming-query and reinforcement-learning based modeling approach for personalized music recommendation
Procedia Comput. Sci.
Multi-modal knowledge-aware reinforcement learning network for explainable recommendation
Knowl. Based Syst.
A deep reinforcement learning based long-term recommender system
Knowl. Based Syst.
Cited by (8)
A deep learning-enabled human-cyber-physical fusion method towards human-robot collaborative assembly
2023, Robotics and Computer-Integrated ManufacturingTowards new-generation human-centric smart manufacturing in Industry 5.0: A systematic review
2023, Advanced Engineering InformaticsA multi-access edge computing enabled framework for the construction of a knowledge-sharing intelligent machine tool swarm in Industry 4.0
2023, Journal of Manufacturing SystemsCitation Excerpt :Cloud brain is responsible for knowledge discovery and knowledge models/data-driven models learning with its powerful computing resources. For knowledge discovery, based on our previous work [32], more than 800 pieces of process knowledge with their domain relationships are mined through the proposed algorithm I and algorithm Ⅱ. The process knowledge is stored in dynamic knowledge bases, which could be continuously updated and accumulated with the increasing of machining data.
Reinforcement Learning system to capture value from Brazilian post-harvest offers
2023, Information Processing in AgricultureA Knowledge-Guided Process Planning Approach with Reinforcement Learning
2024, Research Square