A decentralised multi-agent system for rail freight traffic management

Abstract

The world’s largest coal export operation is located in New South Wales, Australia. The state has more than 87% of the coal transportation done through railways, and one of the strategies to increase throughput is the use of sophisticated computational techniques for rail traffic optimisation. The current state of the art shows a lack of practical applications, thus making scalability, decentralisation and real-world commitment three key research directions. Towards that, this research presents a simulation-based machine learning approach for the railway traffic management problem, in the context of the Hunter Valley Coal Chain (HVCC). We modelled trains, load points and terminals as autonomous intelligent agents that interact, learn and act independently—thus constituting a multi-agent system (MAS). The MAS is implemented on top of a rail network simulation model currently in use at the HVCC. The model is adapted as a decentralised partially-observed Markov decision process environment that allows multi-agent learning via a genetic algorithm. We present experiments with scenarios based on the actual rail network data, which show that the MAS outperforms the heuristic approach embedded in the HVCC simulation tool by up to 81% (in terms of the schedule’s total dwell time). Further to those experiments, a comparison analysis evaluates the relevance of specific state features (e.g. track length, train conflicts, etc.). Finally, an important outcome was that the agents have learned to overcome very complex traffic situations that appear in train scheduling operations and that sometimes result in unnecessarily long dwell times. This type of high level learning represents a significant step forward in the use of complex computational techniques for rail transportation problems.