Authors:
David Kerkkamp
1
;
Zaharah A. Bukhsh
2
;
Yingqian Zhang
2
and
Nils Jansen
1
Affiliations:
1
Radboud University, Nijmegen, The Netherlands
;
2
Eindhoven University of Technology, Eindhoven, The Netherlands
Keyword(s):
Maintenance Planning, Deep Reinforcement Learning, Graph Neural Networks, Sewer Asset Management.
Abstract:
Reinforcement learning (RL) has shown promising performance in several applications such as robotics and
games. However, the use of RL in emerging real-world domains such as smart industry and asset management
remains scarce. This paper addresses the problem of optimal maintenance planning using historical data. We
propose a novel Deep RL (DRL) framework based on Graph Convolutional Networks (GCN) to leverage the
inherent graph structure of typical assets. As demonstrator, we employ an underground sewer pipe network.
In particular, instead of dispersed maintenance actions of individual pipes across the network, the GCN ensures the grouping of maintenance actions of geographically close pipes. We perform experiments using the
distinct physical characteristics, deterioration profiles, and historical data of sewer inspections within an urban
environment. The results show that combining Deep Q-Networks (DQN) with GCN leads to structurally more
reliable networks and a higher degree of ma
intenance grouping, compared to DQN with fully-connected layers
and standard preventive and corrective maintenance strategy that are often adopted in practice. Our approach
shows potential for developing efficient and practical maintenance plans in terms of cost and reliability.
(More)