Research Article  Open Access
Jianwei Lei, Zibin Wang, Hongyuan Fang, Xin Ding, Xiaowang Zhang, Man Yang, Haitao Wang, "Analysis of GPR Wave Propagation in Complex Underground Structures Using CUDAImplemented Conformal FDTD Method", International Journal of Antennas and Propagation, vol. 2019, Article ID 5043028, 11 pages, 2019. https://doi.org/10.1155/2019/5043028
Analysis of GPR Wave Propagation in Complex Underground Structures Using CUDAImplemented Conformal FDTD Method
Abstract
Ground penetrating radar (GPR), as a kind of fast, effective, and nondestructive tool, has been widely applied to nondestructive testing of road quality. The finitedifference timedomain method (FDTD) is the common numerical method studying the GPR wave propagation law in layered structure. However, the numerical accuracy and computational efficiency are not high because of the CourantFriedrichsLewy (CFL) stability condition. In order to improve the accuracy and efficiency of FDTD simulation model, a parallel conformal FDTD algorithm based on graphics processor unit (GPU) acceleration technology and surface conformal technique was developed. The numerical simulation results showed that CUDAimplemented conformal FDTD method could greatly reduce computational time and the pseudowaves generated by the ladder approximation. And the efficiency and accuracy of the proposed method are higher than the traditional FDTD method in simulating GPR wave propagation in twodimensional (2D) complex underground structures.
1. Introduction
GPR is a nondestructive way to detect the internal material distribution of underground structure using electromagnetic waves [1]. Its advantages include being fast, continuous, high resolution and no damage. Because of these advantages, it has been widely applied to geological prospecting, civil engineering, highway reconnaissance, and many other areas [2–6]. Numerical simulation of GPR electromagnetic wave propagation in underground structures is helpful to better explain the real GPR profiles. It can provide a way to explore the link between subsurface structures properties and GPR data.
Various approaches have been developed to simulate GPR wave propagation. These include the raytracing method [7], the transmissionlinematrix method [8], the finite element method (FEM) [9], the semianalytic mode matching algorithm [10], the pseudospectral method [11], the FDTD method [12], the Alternating Direction Implicit (ADI) FDTD method [13], and the symplectic method [14]. Among these, the FDTD method proposed by Yee in 1966 [15] has been the most popular simulation tool due to its great stability and convergence. It uses timestep finitedifference to approximate the differential form of Maxwell’s equations for Faraday and Ampere’s laws and calculates the magnetic and electric field vector components at regularly arranged, distinct points in space. The current state of the art in GPRoriented FDTD approaches has developed rapidly; for example, a software tool called GprMax has been successfully employed in the detection mechanism of GPR [16], a GPUaccelerated FDTD solver integrating it into open source EM simulation software was developed to modeling GPR [17], and a systematic framework for accurate and realistic numerical modeling of GPR for landmine detection has been developed [18]. However, the FDTD method requires large memory and calculation time in simulating GPR wave propagation in 2D complex underground structures due to the limitations of the CFL stability condition [19].
The accuracy and efficiency of forward model are not only related to the performance of the algorithm itself, but also related to computer acceleration technology and modeling method. There are three main methods for improving the accuracy when determining the electromagnetic response of a simulated irregular shape target: one is to increase the density of the discrete grids; however the computation and time also increase correspondingly. For complex problems, the computational resources are difficult to meet. Another approach is to adopt subgrid technology, such that the irregular shape target area uses a subdivision grid, and a normal grid is adopted for other areas. The subgrid technology can greatly improve the calculation accuracy and efficiency, but it is also easy to generate reflection waves at the interface of the coarse and fine grids. A last approach is conformal grid technology [20]. This approach can deal with objects with arbitrary surfaces, and the formula deductions and calculations are simple.
In 2006, NVIDIA developed a generalpurpose parallel computing architecture called compute unified device architecture (CUDA) that leverages parallel compute engine in NVIDIA graphic processor unit (GPU) to solve many complex computational problems [21]. Due to its excellent programmability and remarkable computational power for parallel executions, CUDA has gained tremendous popularity in the scientific computing society. GPU computing has developed in various fields, such as acoustics [22], biomedical applications [23], plasmonics [24], dispersive media [25], and computational electromagnetism [26–34]. FDTD based on the GPU acceleration technique has been applied in the GPR simulation model [35, 36], but the pseudowaves that are generated by the ladder approximation in FDTD modeling method are not considered. This study combines FDTD method and surface conformal technique and GPU acceleration technique proposing a precise and efficient forward modeling algorithm of GPR which can greatly reduce computation time and the pseudowaves that are generated by the ladder approximation.
In this paper, the simulation model of GPR wave propagation in 2D underground structure was established by CUDAimplemented conformal FDTD method. Line source was chosen to be the incident wave, and secondorder Mur absorbing boundary condition was adopted to avoid reflections from the modeled edges. Three numerical examples were provided to verify the high efficiency and accuracy of the CUDAimplemented conformal FDTD method. One obtained the simulated reflection waveform from the twolayer pavement model, the other got the simulated GPR trace profile of the pavement model with structural damage, and the last one demonstrated the GPR simulation section of underground structures with voids.
2. FDTD Method
2.1. The Governing Equations
Among Maxwell’s electromagnetic equations, the FDTD method uses two curl equations:
Here is the electric field, is the magnetic field, ε is the dielectric constant, σ is the conductivity, and μ is the permeability.
Considering 2D Transverse Magnetic (TM) case, (1) can be written as
For 2D TM case, the distribution of the electric and magnetic fields of Yee’s cell of the 2D TM wave is shown in Figure 1 [37].
By applying the central finitedifference approximation to the spatial and time derivatives of (2) yields, the 2D TM wave discrete equations can be written aswhere , .
2.2. Mur Absorbing Boundary Conditions
An absorbing boundary condition allows the FDTD to calculate approximately the infinite domain problems with limited computer capacity. Its basic idea is to truncate boundaries in the computing area through special treatment or add the ideal mathematical absorption material to create the incident wave on the interface of the electromagnetic wave reflection phenomenon. The absorption boundary applied in this paper is the secondorder Mur absorption boundary [38].
For the TM wave, the distribution map of the field component at the left truncated boundary is shown in Figure 2, and the secondorder Mur absorption condition on the left edge can be expressed as
Equation (3) is discrete at point . Using a partial differential in the central difference substitution formula, the resulting difference scheme can be expressed asThe points on other boundaries and corners can be calculated using the same method.
2.3. The Stability Condition
Numerical stability is enforced by the Courant condition, which limits the ratio between the spatial discrete step and time step in the simulation. In 2D grid, this condition is given in the following inequality:where c is the speed of light, Δt is the time step, and Δx and Δy are the spatial step in the x and y direction, respectively.
2.4. Conformal Grid Technology
We adopted a conformal grid method based on effective medium parameters to simulate an arbitrary curved boundary of subsurface structure. A twodimensional circle is shown as an example to explain the conformal grid method theory in Figure 3. Figure 3(a) shows the actual subdivision grid diagram for a circle, the white grids are normal grid points and the green grid is actual subdivision grid of a circle. Figure 3(b) shows the subdivision grid diagram for the conventional ladder approximation of a circle, and the blue grid is conventional ladder approximation grid of a circle. Figure 3(c) shows the conformal grid subdivision diagram for the circle, and the purple grid is conformal grid. The grid points that are entirely inside the circle or entirely outside of the circle are classified as normal grid points, while the rest are classified as conformal grid points. The effective medium parameters are adopted, to process the conformal grids in this paper.
(a)
(b)
(c)
Next, a conformal cell is extracted from Figure 3(c) to illustrate the effective medium parameters for the conformal grid points in the twodimensional TM wave example. In the FDTD methods, the electric field E is located at the midpoint of the conformal cell, and the magnetic field H is located at the edge of the conformal cell. The calculation for a conformal grid point of the medium in Cartesian coordinates is shown in Figure 4, where N is the sampling point of the field components ; A and B are the sampling points of the field components ; C and D are the electric field sample points of the field components . Δx and Δy are the width and height of a grid, and and are the length of the different media on a corresponding edge; and are the area of media 1 and 2, respectively. We assume that the electromagnetic parameters of media 1 and 2 are ε_{1}, σ_{1}, μ_{1} and ε_{2}, σ_{2}, μ_{2}, respectively.
As the magnetic field node is located at the middle point of the grid edge, the effective values of the magnetic conductivity and permeability at each magnetic field node can be calculated by a weighted average of the length of the different media on a corresponding edge. Similarly, the effective values of the dielectric constant and conductivity at each electric field node can be calculated by a weighted average of the area occupied by different media on the corresponding surface. The effective magnetic permeability of the A, B, C, and D nodes in Figure 4(b) can be expressed as
The effective magnetic permeability of the conformal mesh, respectively, can be written as
The effective dielectric constant and the effective conductivity of the N point in Figure 4(b) can be shown aswhere eff denotes the equivalent value of the parameter.
Substituting (7)(9) into (3), we obtain the differential iterative equations for the conformal FDTD method:where , .
3. The Simulation of GPR Waves on GPU
3.1. CUDAImplemented FDTD Method
CUDA programming is well suited to solve problems that can be expressed as parallel computing, which achieves application acceleration performance by mapping data elements to multiple threads. The kernel is a function that is executed on the device, which is the most important component of the CUDA program. The kernel is always represented as a sequential program on the host and executed on the device. CUDA exposes a twolevel thread hierarchy that provides an optimization method for developers. All threads are divided into blocks, all of which are grouped into a single grid. The kernel executes in the grid of thread blocks.
E and H field components calculate in a bidimensional xy square domain. The number of threads is decided by the calculation domain, which is the number of Yee units, because we map threads to units. As shown in Figure 5, the strategy adopted is mapping each thread to Yee’s cell and each thread block to a group of cells. Each thread deals with one Yee cell during the parallel computing. In this way, the entire calculation domain is wrapped by the CUDA grid.
In 2D case, the flowchart of CUDA kernel performing the field calculation is depicted in Figure 6 [39]. There are two CUDA kernels performing the field calculations. The first kernel computes E, and the second updates H. T is time step; is total time step.
3.2. The Forward Modeling
The steps of the forward modeling of the propagation of GPR in the layer pavement system are as follows:(1)Input basic parameters of pavement structure and build pavement structure model(2)Obtain and input the source of excitation; make numerical difference a discrete source and make it a point corresponding to the time step(3)Calculate the propagation characteristics of GPR electromagnetic waves in the pavement structure with the finitedifference principle of FDTD, including reflection, loss characteristics, analysis, and calculation of the general field(4)Calculate the reflection field(5)Output the signal of radar reflection
According to the above steps, a forward model of the GPR propagation in the pavement structure is established. The block diagram of the program is shown in Figure 7.
3.3. The Accuracy of the Proposed Algorithm
The following numerical simulations were performed on a desktop computer with Intel Core i76700K CPU and NVIDIA Geforce GTX 1070 Graphics card. The desktop operating system is Windows 7 Pro 64bit. The development tool of CUDA program is the CUDA Toolkit 7.5 built with Microsoft Visual Studio 2010. In the following, we verified the accuracy of the absorption boundary and algorithm by a simple example.
In the 2D FDTD region in free space, the source of excitation was a sine point source E(t) = sin(2π · t · dt · m) (m= 1.5e13), and Mur boundary was used at the regional truncation boundary. In order to ensure the stability and convergence of FDTD method, the spatial discrete grid and the time step were chosen as Δx = Δy = 0.25 cm and dt = 0.005 ns, respectively. The point source was initiated in the center of the 50 cm × 50 cm square free space by using the 2D TM wave as an example.
A CUDA calculation model was established for each thread to solve a Yee cellular (magnetic) field. There were 16 × 16 threads in each block, and the simulated area was divided into 200 × 200 grids. The point source and observation point were located at (25 cm, 25 cm) and (25 cm, 30 cm), respectively. Using traditional FDTD method and CUDAimplemented FDTD method, we calculated the 600timestep field change with the same parameter calculation. As shown in Figure 8, the simulation results of traditional FDTD method and CUDAimplemented FDTD method were with excellent agreement.
As shown in Figure 9, it was the 600timestep field change diagram calculated by the CUDAimplemented FDTD method and traditional FDTD method. It can be seen that GPR electromagnetic wave reached the boundary without causing reflection due to the setting of absorbing boundary. It also simulated intuitively the GPR electromagnetic wave propagation in free space and proved the accuracy of the Mur absorbing boundary.
(a)
(b)
4. Numerical Simulation
Firstly, we verified the high efficiency of the CUDAimplemented FDTD method by simulating a twolayer pavement model. As shown in Figure 10, the top layer of the model represented air with = 1, σ = 0; the second layer represented asphalt concrete with = 6, σ = 1 mS / m; the third layer was the base with = 30 and σ = 2 mS / m. We assumed that all materials are nonmagnetic. The time step and spatial increment were set equal to dt = 0.005 ns and Δx = Δy = dl = 0.25 cm, respectively. We used a Ricker wavelet with a unit amplitude and a center frequency of 1.0 GHz (Figure 11) to represent the line source of the GPR transmitter.
The transmitter was at the point (15 cm, 100 cm) and the receiver was at the point (15 cm, 110 cm), respectively. The simulation results of traditional FDTD method and CUDAimplemented FDTD method after 1000 time steps were shown in Figure 12. Excellent agreement was reached. It was proved that traditional FDTD method and CUDAimplemented FDTD method have the same accuracy. Moreover, the CUDAimplemented FDTD method consumed 26.276 s, while the traditional FDTD method increased to 299.919 s. The former saved 91.24% of the time. The result demonstrated that CUDAimplemented FDTD method greatly reduced computation time.
As shown in Figure 12, it can prove that CUDAimplemented FDTD method has the same accuracy as the traditional FDTD method. The speedup ratios achieved are shown in Table 1. When the number of units is small, the acceleration ratio is also small, so the acceleration performance obtained is not significant. When the number of units is getting larger, better acceleration performance will be obtained. In particular, when the FDTD cells are up to 4096×4096, we can obtain more than 13 times speedup ratio.

GPR profiles of a structurally damaged pavement were simulated in second example. As shown in Figure 13, the first layer of the model was an air layer, and the bottom was threelayer pavement with structural damage. There was a 10 mm wide crack in first layer. There were a circular nodense region with radius of 5 cm and a 30 cm × 40 cm rectangular cavity in second layer. In this example, assuming that the porosity of the region was 10%, all dielectric parameters of the material were shown in Figure 13. Receivers and transmitters were placed along the airearth interface every 1 cm to measure the reflection waveform of the GPR wave in the pavement structure. The distance between receiver and transmitter was 5 cm. The incident wave in first example was used to represent the line source of the transmitter in this model. The increments of the time step and spatial interval were selected as dt = 0.01 ns and Δx = Δy = dl = 0.5 cm, respectively. GPR tracking profiles of the model simulated by CUDAimplemented FDTD method and traditional FDTD method were plotted in Figures 14 and 15.
As shown in Figures 14 and 15, a good agreement has been reached. However, the traditional FDTD method required 2.7357 h; the CUDAimplemented FDTD method required 0.1771 h. The results showed that the reflection waveform of CUDAimplemented FDTD method was consistent with the traditional FDTD method, but the CUDAimplemented method reduced the computational time by more than 93.5%.
In this example, a complex underground structure with two cavities was established. We verified the high efficiency and accuracy of CUDAimplemented conformal FDTD method by using conformal and nonconformal modeling method for two cavities, respectively. As shown in Figure 16, this model included an airearth interface and a soil layer with two cavities with a radius of 5 cm. The model size and dielectric parameters were shown in Figure 16. The excitation source was the same as the second model. The time step and spatial interval increment were selected as dt = 0.01 ns and Δx = Δy = dl = 0.5 cm, respectively.
We placed receivers and transmitters along the airearth interface every 1 cm. The distance between the receiver and transmitter was 10 cm. Figures 17 and 18 were GPR simulation section using CUDAimplemented FDTD method and traditional FDTD method. The right cavity used conformal modeling method, the left one with nonconformal modeling method. And from Figures 17 and 18, it can be seen that the conformal modeling method has fewer pseudowaves than the nonconformal modeling method. The CUDAimplemented FDTD method required 967.23 s, and traditional FDTD method required 11606.76 s. The results indicate that CUDAimplemented conformal FDTD algorithm can greatly reduce computational time and the pseudowaves that are generated by the ladder approximation.
5. Conclusions
This paper presented a CUDAimplemented conformal FDTD method for simulating GPR wave propagation in 2D lowloss medium. The high efficiency and accuracy of the method were verified by three numerical examples. The results showed that CUDAimplemented conformal FDTD algorithm can greatly reduce computation time and the pseudowaves that were generated by the ladder approximation. In addition, we obtained the GPR simulation profile of complex underground structure by forward modeling using the CUDAimplemented conformal FDTD method. By analyzing the simulated profiles of GPR, we can deepen our understanding of the propagation of GPR waves in underground space and better interpret the real GPR profiles. This paper focuses on 2D isotropic conditions. However, this method can be extended to threedimensional (3D) or anisotropic situations. Through further research, we will develop a CUDAimplemented conformal FDTD to simulate the propagation of GPR waves in anisotropic media or 3D structures.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This research was supported by the National Natural Science Foundation of China (No. 51678536, No. 41404096), the National Key Research and Development Program of China (No. 2017YFC1501200), the Scientific and Technological Research Program of Henan Province (No. 152102310066), and the Henan Transportation Science and Technology Project of Henan Province in 2018 (No. 2018J7) for which the authors are grateful.
References
 Q.W. Dai, D.S. Feng, and J.S. He, “Finite difference time domain method forward simulation of complex geoelectricity ground penetrating radar model,” Journal of Central South University of Technology (English Edition), vol. 12, no. 4, pp. 478–482, 2005. View at: Publisher Site  Google Scholar
 N. J. Cassidy and T. M. Millington, “The application of finitedifference timedomain modelling for the assessment of GPR in magnetically lossy materials,” Journal of Applied Geophysics, vol. 67, no. 4, pp. 296–308, 2009. View at: Publisher Site  Google Scholar
 U. Böniger and J. Tronicke, “Integrated data analysis at an archaeological site: A case study using 3D GPR, magnetic, and highresolution topographic data,” Geophysics, vol. 75, no. 4, pp. B169–B176, 2010. View at: Publisher Site  Google Scholar
 Z. Zyada, T. Matsuno, Y. Hasegawa, S. Sato, and T. Fukuda, “Advances in GPRbased landmine automatic detection,” Journal of The Franklin Institute, vol. 348, no. 1, pp. 66–78, 2011. View at: Publisher Site  Google Scholar
 C. Anitescu, Y. Jia, Y. J. Zhang, and T. Rabczuk, “An isogeometric collocation method using superconvergent points,” Computer Methods Applied Mechanics and Engineering, vol. 284, pp. 1073–1097, 2015. View at: Publisher Site  Google Scholar
 K. M. Hamdia, H. Ghasemi, X. Zhuang, N. Alajlan, and T. Rabczuk, “Sensitivity and uncertainty analysis for flexoelectric nanostructures,” Computer Methods Applied Mechanics and Engineering, vol. 337, pp. 95–109, 2018. View at: Publisher Site  Google Scholar
 X. Zeng, G. A. McMechan, J. Cai, and H. W. Chen, “Comparison of ray and Fourier methods for modeling monostatic ground penetrating radar profiles,” Geophysics, vol. 60, no. 6, pp. 1727–1734, 1995. View at: Publisher Site  Google Scholar
 C. Liu and L. Shen, “Numerical simulation of subsurface radar for detecting buried pipes,” IEEE Transactions on Geoscience and Remote Sensing, vol. 29, no. 5, pp. 795–798. View at: Publisher Site  Google Scholar
 A. Villarino, B. Riveiro, D. GonzalezAguilera, and L. J. SánchezAparicio, “The integration of geotechnologies in the evaluation of a wine cellar structure through the finite element method,” Remote Sensing, vol. 6, no. 11, pp. 11107–11126, 2014. View at: Publisher Site  Google Scholar
 A. W. Morgenthaler and C. M. Rappaport, “GPR wave scattering from complex objects using the semianalytic mode matching algorithm: coordinate scattering center selection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 6, pp. 1949–1956, 2011. View at: Publisher Site  Google Scholar
 J. M. Carcione, S. Valle, and G. Lenzi, “GPR modelling by the Fourier method: improvement of the algorithm,” Geophysical Prospecting, vol. 47, no. 6, pp. 1015–1029, 1999. View at: Publisher Site  Google Scholar
 E. Karpat, M. Çakır, and L. Sevgi, “Subsurface imaging, FDTDbased simulations and alternative scan/processing approaches,” Microwave and Optical Technology Letters, vol. 51, no. 4, pp. 1070–1075, 2009. View at: Publisher Site  Google Scholar
 D.S. Feng and Q.W. Dai, “GPR numerical simulation of full wave field based on UPML boundary condition of ADIFDTD,” NDT & E International, vol. 44, no. 6, pp. 495–504, 2011. View at: Publisher Site  Google Scholar
 T. Hirono, W. W. Lui, and K. Yokoyama, “Timedomain simulation of electromagnetic field using a symplectic integrator,” IEEE Microwave and Guided Wave Letters, vol. 7, no. 9, pp. 279–281, 1997. View at: Publisher Site  Google Scholar
 K. S. Yee, “Numerical solution of initial boundary value problems involving Maxwell equations in isotropic meidia,” IEEE Transactions on Antennas and Propagation, vol. 14, no. 3, pp. 302–307, 1966. View at: Publisher Site  Google Scholar
 A. Giannopoulos, “Modelling ground penetrating radar by GprMax,” Construction and Building Materials, vol. 19, no. 10, pp. 755–762, 2005. View at: Publisher Site  Google Scholar
 C. Warren, A. Giannopoulos, A. Gray et al., “A CUDAbased GPU engine for gprMax: Open source FDTD electromagnetic simulation software,” Computer Physics Communications, vol. 237, pp. 208–218, 2019. View at: Publisher Site  Google Scholar
 I. Giannakis, A. Giannopoulos, and C. Warren, “A realistic FDTD numerical modeling framework of ground penetrating radar for landmine detection,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 9, no. 1, pp. 37–51, 2016. View at: Publisher Site  Google Scholar
 A. Taflove, Computational Electrodynamics: The FiniteDifference TimeDomain Method, Artech House, Norwood, MA, USA, 3rd edition, 2005. View at: MathSciNet
 X.J. Hu, D.B. Ge, and B. Wei, “Study on MCFDTD for 3D coated targets by using effective parameters,” Journal of Systems Engineering and Electronics, vol. 28, no. 11, pp. 1652–1667, 2006. View at: Google Scholar
 E. Lindholm, J. Nickolls, S. Oberman, and J. Montrym, “NVIDIA Tesla: a unified graphics and computing architecture,” IEEE Micro, vol. 28, no. 2, pp. 39–55, 2008. View at: Publisher Site  Google Scholar
 J. J. Lopez, D. Carnicero, N. Ferrando, and J. Escolano, “Parallelization of the finitedifference timedomain method for room acoustics modelling based on CUDA,” Mathematical and Computer Modelling, vol. 57, no. 7, pp. 1822–1831, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 . Jieru Chi, . Feng Liu, E. Weber, . Yu Li, and S. Crozier, “GPUaccelerated FDTD modeling of radiofrequency field–tissue interactions in highfield MRI,” IEEE Transactions on Biomedical Engineering, vol. 58, no. 6, pp. 1789–1796, 2011. View at: Publisher Site  Google Scholar
 K. H. Lee, I. Ahmed, R. S. Goh, E. H. Khoo, E. P. Li, and T. G. Hung, “Implementation of the fdtd method based on lorentzdrude dispersive model on gpu for plasmonics applications,” Progress in Electromagnetics Research, vol. 116, pp. 441–456, 2011. View at: Publisher Site  Google Scholar
 M. R. Zunoubi, J. Payne, and W. P. Roach, “CUDA implementation of TEZFDTD solution of Maxwell's equations in dispersive media,” IEEE Antennas and Wireless Propagation Letters, vol. 9, pp. 756–759, 2010. View at: Publisher Site  Google Scholar
 N. Takada, N. Masuda, T. Tanaka, Y. Abe, and T. Ito, “A GPU implementation of the 2D finitedifference timedomain code using high level shader language,” Applied Computational Electromagnetics Society Journal, vol. 23, no. 4, pp. 309–316, 2008. View at: Google Scholar
 A. Dziekonski, A. Lamecki, and M. Mrozowski, “GPU acceleration of multilevel solvers for analysis of microwave components with finite element method,” IEEE Microwave and Wireless Components Letters, vol. 21, no. 1, pp. 1–3, 2011. View at: Publisher Site  Google Scholar
 M. Livesey, J. F. Stack, F. Costen, T. Nanri, N. Nakashima, and S. Fujino, “Development of a CUDA implementation of the 3D FDTD method,” IEEE Antennas and Propagation Magazine, vol. 54, no. 5, pp. 186–195, 2012. View at: Publisher Site  Google Scholar
 R. Shao, D. Linton, I. Spence, P. Milligan, and N. Zheng, “Detection and GPU acceleration of 3D FDTD algorithms based on memory access patterns,” in Proceedings of the 2013 International Conference on Mechatronic Sciences, Electric Engineering and Computer, MEC 2013, pp. 2520–2526, China, December 2013. View at: Google Scholar
 T. T. Zygiridis, N. V. Kantartzis, and T. D. Tsiboukis, “GPUaccelerated efficient implementation of FDTD methods with optimum timestep selection,” IEEE Transactions on Magnetics, vol. 50, no. 2, 2014. View at: Google Scholar
 J. Francés, B. Otero, S. Bleda et al., “MultiGPU and multiCPU accelerated FDTD scheme for vibroacoustic applications,” Computer Physics Communications, vol. 191, pp. 43–51, 2015. View at: Publisher Site  Google Scholar
 S. Nanthakumar, T. Lahmer, X. Zhuang, G. Zi, and T. Rabczuk, “Detection of material interfaces using a regularized level set method in piezoelectric structures,” Inverse Problems in Science and Engineering, pp. 1–24, 2015. View at: Publisher Site  Google Scholar
 M. Cicuttin, L. Codecasa, B. Kapidani, R. Specogna, and F. Trevisan, “GPU accelerated timedomain discrete geometric approach method for maxwell's equations on tetrahedral grids,” IEEE Transactions on Magnetics, vol. 54, no. 3, 2018. View at: Google Scholar
 H. Ghasemi, H. S. Park, and T. Rabczuk, “A multimaterial level setbased topology optimization of flexoelectric composites,” Computer Methods Applied Mechanics and Engineering, vol. 332, pp. 47–62, 2018. View at: Publisher Site  Google Scholar
 T. M. Millington and N. J. Cassidy, “Optimising GPR modelling: a practical, multithreaded approach to 3D FDTD numerical modelling,” Journal of Computers & Geosciences, vol. 36, no. 9, pp. 1135–1144, 2010. View at: Publisher Site  Google Scholar
 I. Giannakis, A. Giannopoulos, and C. Warren, “Realistic FDTD GPR antenna models optimized using a novel linear/nonlinear fullwaveform inversion,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 3, pp. 1768–1778, 2019. View at: Publisher Site  Google Scholar
 R. Luebbers, D. Ryan, and J. Beggs, “A twodimensional timedomain nearzone to farzone transformation,” IEEE Transactions on Antennas and Propagation, vol. 40, no. 7, pp. 848–851. View at: Publisher Site  Google Scholar
 G. Mur, “Absorbing boundary conditions for the finitedifference approximation of the timedomain electromagneticfield equations,” IEEE Transactions on Electromagnetic Compatibility, vol. 23, no. 4, pp. 377–382, 1981. View at: Publisher Site  Google Scholar
 X. Wang, S. Liu, X. Li, and S. Zhong, “GPUaccelerated finitedifference timedomain method for dielectric media based on CUDA,” International Journal of RF and Microwave ComputerAided Engineering, vol. 26, no. 6, pp. 512–518, 2016. View at: Publisher Site  Google Scholar
Copyright
Copyright © 2019 Jianwei Lei et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.