Optimising GPR modelling: A practical, multi-threaded approach to 3D FDTD numerical modelling

Abstract

The demand for advanced interpretational tools has lead to the development of highly sophisticated, computationally demanding, 3D GPR processing and modelling techniques. Many of these methods solve very large problems with stepwise methods that utilise numerically similar functions within iterative computational loops. Problems of this nature are readily parallelised by splitting the computational domain into smaller, independent chunks for direct use on cluster-style, multi-processor supercomputers. Unfortunately, the implications of running such facilities, as well as time investment needed to develop the parallel codes, means that for most researchers, the use of these advanced methods is too impractical. In this paper, we propose an alternative method of parallelisation which exploits the capabilities of the modern multi-core processors (upon which today’s desktop PCs are built) by multi-threading the calculation of a problem’s individual sub-solutions.

To illustrate the approach, we have applied it to an advanced, 3D, finite-difference time-domain (FDTD) GPR modelling tool in which the calculation of the individual vector field components is multi-threaded. To be of practical use, the FDTD scheme must be able to deliver accurate results with short execution times and we, therefore, show that the performance benefits of our approach can deliver runtimes less than half those of the more conventional, serial programming techniques. We evaluate implementations of the technique using different programming languages (e.g., Matlab, Java, C++), which will facilitate the construction of a flexible modelling tool for use in future GPR research. The implementations are compared on a variety of typical hardware platforms, having between one and eight processing cores available, and also a modern Graphical Processing Unit (GPU)-based computer. Our results show that a multi-threaded xyz modelling approach is easy to implement and delivers excellent results when implemented in a practical, Java-coded modelling tool.

Introduction

Ground-penetrating radar (GPR) is, arguably, the most popular, broad-based geophysical technique for near-surface ground investigation due to its unrivalled ability to stratigraphically map large areas in relatively short times. Unfortunately, the processing and interpretation of GPR data can be both difficult and time consuming, particularly in complex, heterogeneous environments. In the past few years, advanced GPR processing, modelling and interpretation methods have lead to more sophisticated approaches to data analysis and the ability to ‘extract’ material property information from the GPR sections, such as water content, porosity, etc. (Galagedara et al., 2005, Galagedara et al., 2003; Reppert et al., 2000). Sophisticated analysis and simulation techniques, such as finite-difference time-domain modelling and tomographic inversion, are now being applied to applications such as land contamination (Cassidy, 2008, Cassidy, 2007a; de Castro and Branco, 2007), hydrology (Kuroda et al., 2007) engineering (Liu et al., 2007) and utility/landmine detection (Crocco et al., 2009, Lopera and Milisavljevic, 2007). In part, this is due to the complexity of interpreting data generated from the latest GPR system designs, (e.g., multi-antenna array systems—Yarovoy et al., 2007), but also a consequence of our improved ability to accurately simulate, or model, GPR wave propagation at realistic, three-dimensional scales and in sensible times frames, measured in hours rather than of days (Cassidy, 2007b). The fundamental principles governing the propagation of the electromagnetic (EM) radiation are well known and have been successfully embedded into forward modelling computer programs for a wide range of applications since the late 1970s (Taflove and Hagness, 2005). For the forward modelling of ground penetrating radar, the method of moments (MoM) and, more specifically, the finite-difference time-domain (FDTD) technique have become the most popular simulation tools with both approaches being used to replicate practical near-surface GPR surveys in a range of environments (Cassidy, 2007a, Giannopoulos, 2009). As with all computationally demanding modelling schemes, any improvement in efficiency and program speed is of ultimate benefit to the user and with the increased use of multi-threaded parallel sequencing in modern PC hardware/software, it is now feasible to run complex, ‘realistic’ scale 3D GPR models on high-end portable and desktop machines.

In this paper, we focus on the application of the finite-difference time-domain (FDTD) technique for GPR wave propagation modelling and present an optimised multi-threaded approach to the core 3D algorithm that offers significant performance improvement over conventional sequential implementations. We begin, in Section 2, by explaining how multi-threading can be used to exploit modern multi-core CPU technology. Then, in Section 3, we introduce the fundamentals of the FDTD modelling scheme and discuss ways in which the calculation effort can be divided into thread-safe sub-tasks. In Section 4, we compare the different implementations’ performance by carrying out a simplified modelling task using C++, Java and Matlab, across four different platforms:

• Dual core machine (32 bit Intel^® Core™ 2 Duo T7500 at 2.20 GHz),

• Twin dual core machine (64 bit AMD^® Opteron™ 280, 2 X dual core, 2.8 GHz),

• Quad core machine (32 bit Intel^® Core™ 2 Quad Q9300 at 2.5 GHz),

• Twin quad core machine (Intel 64 bit Xeon 5320, 1.86 GHz).

Having observed the increasing exploitation of the massively parallel capabilities of modern Graphics Processing Units by researchers in the scientific programming community (Adams et al., 2007), we have also added a GPU-specific implementation of the FDTD code executed via the quad-core machine on a modern graphics card (nVidia GeForce 8800 GT). Finally, in Section 5, we illustrate the use of the multi-threaded approach in an advanced, flexible 3D GPR modeller before drawing our conclusions in Section 6.