Solving Partial Differential Equations Using a New Differential Evolution Algorithm

Panagant, Natee; Bureerat, Sujin

doi:https://doi.org/10.1155/2014/747490

Mathematical Problems in Engineering

On this page

Abstract Introduction Results and Discussion Conclusions References Copyright Related Articles

Research Article | Open Access

Volume 2014 | Article ID 747490 | https://doi.org/10.1155/2014/747490

Solving Partial Differential Equations Using a New Differential Evolution Algorithm

Natee Panagant¹and Sujin Bureerat¹

Academic Editor: Khai Ching Ng

Received07 Feb 2014

Accepted26 May 2014

Published12 Jun 2014

Abstract

This paper proposes an alternative meshless approach to solve partial differential equations (PDEs). With a global approximate function being defined, a partial differential equation problem is converted into an optimisation problem with equality constraints from PDE boundary conditions. An evolutionary algorithm (EA) is employed to search for the optimum solution. For this approach, the most difficult task is the low convergence rate of EA which consequently results in poor PDE solution approximation. However, its attractiveness remains due to the nature of a soft computing technique in EA. The algorithm can be used to tackle almost any kind of optimisation problem with simple evolutionary operation, which means it is mathematically simpler to use. A new efficient differential evolution (DE) is presented and used to solve a number of the partial differential equations. The results obtained are illustrated and compared with exact solutions. It is shown that the proposed method has a potential to be a future meshless tool provided that the search performance of EA is greatly enhanced.

1. Introduction

In the field of computational mechanics, many engineering systems can be viewed as continuum domains and their physics are represented by using differential equations, for example, heat transfer, structures, and fluid mechanics. Some of such differential equations can be solved analytically; however, in most practical cases, the domains are too complicated for an analytical approach. This leads to the creation of numerical approximation of the differential equations. Some of the established approaches include finite element methods [1–4], finite different methods [5, 6], and finite volume methods [7–9]. Those previously mentioned methods are classified as the mesh-based or grid-based methods. The concept is to approximate a complicated domain of differential equations with a finite number of simple grids enabling one to find the approximate solutions of the differential equations.

Nevertheless, although the mesh-based methods like finite element analysis have become a traditional engineering tool with numerous commercial software packages being developed around the world, they still have some drawbacks, for example, difficulties in mesh generation and mesh distortion in large deformation analysis of structures. This causes researchers and engineers to search for alternative numerical approaches and the so-called meshless methods were originated. The method unlike the traditional finite element or finite volume methods uses only distribution of nodes on a differential equation domain to achieve its approximate solution. Over the last few decades, there have been numerous meshless methods proposed to solve a number of engineering applications. Starting with one of the earliest methods, the smooth particle hydrodynamics [10, 11] approach; then, there have been other established methods such as the element-free Galerkin method [12], the reproducing kernel particle method [13], and the local Petrov-Galerkin method [14–16]. A good review on meshless methods can be found in [17, 18]. For some up-to-date development, see [19–22]. The meshless methods, despite having some advantages, have some weak points such as difficulties in dealing with boundary conditions (BCs). Therefore, more research and ideas are still needed in this field.

This paper proposes an alternative way to achieve solutions of partial differential equations (PDEs). The approach is classified as a meshless method since only nodes are generated on a differential equation domain. A partial differential equation problem is converted into an optimisation problem while a metaheuristic (MH) or an evolutionary algorithm (EA) is employed to search for the optimum solution. Such an idea has been presented in [23, 24] but there are many obstacles to overcome if one wants to use it in practice. The most important issue is the low convergence rate of EAs which consequently results in a not-so-good approximate solution. However, the attractiveness of this approach remains due to the nature of a soft computing technique in EA. The algorithm can be used to tackle almost any kind of optimisation problem with simple random-directed evolutionary operation, which means it is mathematically simpler to use. In this paper, a new efficient differential evolution (DE) is presented and used to solve a number of the partial differential equations. The results obtained are illustrated and compared with exact solutions. The rest of this paper is organised as follows. The second section briefly reviews the differential evolution method where the new DE is proposed. Section 3 details how to solve the partial differential equations by means of evolutionary optimisation. The results are shown and discussed in Section 4 while conclusions are drawn in Section 5.

2. Differential Evolution Algorithm

Some of the earliest evolutionary algorithms were proposed a few decades ago where genetic algorithms are arguably the best known method. Since then, this kind of soft computing has gained increasing popularity year after year because of their simplicity to use, derivative-free feature, and robustness. So far, there have been a thousand of EAs and their variants developed and presented in research literature. Among such an incredibly large number of algorithms, differential evolution is one of the top performers for general optimisation purposes. The method is a parallel random-directed search method that was first proposed by Storn and Price [25]. DE is powerful yet very simple to understand, code, and implement which resulted in its great popularity nowadays. A large number of DE variants have been proposed, for example, in [26, 27]. Similar to most EAs, a DE computational procedure starts with a randomly generated initial real-code population. Then the DE operators (mutation, crossover, and selection) are iteratively activated to improve the population approaching to an optimum. DE since it was first introduced has various schemes. In this paper, we use 4 versions of the original differential evolution algorithm: DE/rand/1/bin, DE/rand/1/exp, DE/current-to-best/2/bin, and DE/current-to-best/2/exp. Note that the notation of DE schemes is described as where is the vector to be mutated (“rand” for a randomly chosen vector and “current-to-best” for a current population vector that is mutated by the best so far vector), is a number of pair of different vectors used in the mutation operation, and is a crossover type (“bin” for binomial crossover and “exp” for exponential crossover).

2.1. Differential Evolution Operators

There are three main operators for DE which can be briefly detailed as follows.

2.1.1. Mutation

For a population vector , , where is a generation number and is a population size, a mutant vector for the rand scheme is produced using (2) and for the current-to-best scheme is produced using (3). Consider where , , and are random integer indices of population vectors (), is current population vector of last generation, is a population vector that has the best (minimum) fitness in the previous generation, is a scaling factor () that controls amplification of difference vectors, and is an additional control variable () that controls influence of the best population vector.

2.1.2. Crossover

In the crossover operation, a target vector () will be bred with a mutant vector () leading to a trial vector (). The probability of crossover () will be predefined. Once the operator is revoked, there can be two types of crossover as binomial and exponential crossovers which are given in Algorithms 1 and 2, respectively, where CR is a prespecified constant and is a dimension of a design vector.

Input: x, v, CR
Output u
(1) Set u = x
(2) Randomly choose
(3) For to
(4) Generate a uniform random number 0, 1.
(5) If rand ≤ CR or , set .
(6) Next

Input: x, v, CR
Output u
(1) Set u = x
(2) Randomly choose
(3) Set . Generate a uniform random number 0, 1.
(4) While rand ≤ or , do
(5) ,
(6) , If ,
(7) End while

2.1.3. Selection

Before adding trial vectors into a new generation population, the selection operator will compare the fitness function value of a trial vector to that of its target vector. Then the vector that has a lower fitness function value is selected into the new generation population.

2.1.4. Termination Criteria

In this paper, we specify two stopping criteria. The calculation will be terminated when maximum function evaluation is reached or allowable minimum fitness function and constraint values are reached.

2.2. New Differential Evolution Algorithm

In this work, we present new DE with variation of population size during an optimisation run. This DE scheme will be termed DE-New. The differential operators (Mutation, crossover, and selection) and stopping criteria of DE-New are the same as those used in the original DE. The new DE starts searching with an initial population size. The modification is based on the premise that a larger population size tends to produce some better design solutions. Nevertheless, using a large population size throughout the optimisation run can be time consuming. Thus, the concept of population size variation is employed. As the algorithm continues, if a convergence speed of the algorithm at the current iteration is not satisfied, then additional members will be generated and added to the population. The flowchart of DE-New is shown in Figure 1.

During the search process in which the best fitness for all iterations are obtained, if the standard deviation of the best fitness values of the previous npv iterations is less than a predefined value pvc, then the number of solution members in a population is increased but it must be limited in such a way that the total number of population members must not exceed a predefined maximum population size. In cases that the current population size is less than maximum population size and the aforementioned criterion is met, additional solution vectors will be generated and added to the population. Given that NP is the size of the current population, other NP solutions in which half of them are randomly generated (exploration) and the rest are created by means of DE operation (exploitation) will be added to the population. Note that, in this paper, DE/rand/1/exp and DE/current-to-best/2/exp are used with this population variation concept. The DE/rand/1/exp scheme is used to update a regular population for each iteration while the DE/current-to-best/2/exp scheme is used in the additional population generation process.

3. Numerical Examples

The proposed approach is somewhat similar to the literature [24] in the sense that a partial differential equation problem is altered to be a minimisation problem with equality constraints. Initially, nodes are generated inside and on the boundary of a PDE domain. A function for approximating a PDE solution is selected where boundary conditions are treated as equality constraints. The optimisation problem is posed to find function coefficients in such a way to minimise residuals or square errors. In the previous work, evolution strategies (ES) were used as an optimiser while an approximate function is a partial sum of Fourier cosine series. In this work, DE will be used instead of ES as it is found to be more powerful whereas a polynomial function is used to estimate the PDE solution. In order to demonstrate the performance of a new approach, 6 PDE problems are used for numerical test which can be detailed as follows: PDE1: governing equation: , domain: , BCs: ; ; ; and , exact solution: ; PDE2: governing equation: , domain: , BCs: ; ; ; and , exact solution: ; PDE3: governing equation: , domain: , BCs: ; ; ; and , exact solution: ; PDE4: governing equation: , domain: , BCs: ; ; ; and , exact solution: ; PDE5: governing equation: , domain: , BCs in , exact solution: ; PDE6: governing equation: , domain: , BCs: in , exact solution: .The notation herein is a Laplacian operator. The acronym PDE means the th partial differential equation problem while BC means boundary condition. PDE solutions are approximated by a polynomial expansion as in (4). Consider where are polynomial coefficients to be determined and is a maximum of the polynomial order for and .

The number of approximation terms is , which means that there are design variables for an optimisation problem. Since (4) is in a polynomial form, we can calculate any orders of derivatives, which is simple to be used with a wide range of differential equations.

A typical partial differential equation can be written in a general form as with BCs on for .

Let be the number of nodes being assigned on a PDE domain and boundary where is the number of nodes on the boundaries . If we substitute (4) into a PDE for all points on , we can have equations with unknowns. Similarly, by substituting (4) on the BCs for all points on , we have equality constraints. This implies that we convert the PDE (5) to be a constrained optimisation problem where the equations is replaced by an objective function to be minimised [23] which is calculated by means of a weighted residual approach. The minimisation problem can then be posed as which is subject to where is a vector of design variables or polynomial coefficients, is a residual function, is the weight function, in this case, which is set as unity as used in [23], is an area of the domain, and are obtained from substituting (4) into the boundary conditions for all nodes on the boundaries.

Figure 2 shows how to compute the integration (6) on a particular subarea which is discretised by the nodes on the domain. The value of the residual function on the subarea is the average value of from those 4 points. The ideal solution is to have the objective function WRF becoming zero.

In order to handle the equality constraints , a penalty function is used which can be expressed as where are penalty parameters (set to be 50 in this study).

Thus, the fitness function (FFV) for DE can be calculated as The accuracy of the final solutions is measured using a root mean squared error (RMSE) which can be computed as where is an approximate function value at node , is the exact function value at node , and is the number of nodes.

Five versions of DE (DE/rand/1/bin, DE/rand/1/exp, DE/current-to-best/2/bin, DE/current-to-best/2/exp, and DE-New) are employed to tackle those six optimisation problems. The optimisation parameter settings are given as , , , and . An optimiser is stopped when either the maximum number of function evaluations (500,000) is reached or the objective and penalty function values are less than 0.001. The best solution of the last generation of DE is classified as an optimum. Population variation criteria are npv = 10 and pvc = 0.001. The population size for the original DE is set to be 400 while the initial population size for DE-New is set as 100. The maximum population size for DE-New is 400. For all PDE problems, the number of nodes is 100. Nodes are equispaced throughout the square domains for PDE1-4. The nodal distributions for PDE5-6 are illustrated in Figures 3 and 4, respectively.

4. Results and Discussion

The numerical experiment was carried out on a personal computer where the program is written using MATLAB computing language. Having employed five DE strategies for tackling six optimisation problems of PDE1-6, the results obtained are compared in Table 1. In the table, there are two indicators for comparison, the fitness function value calculated using (5) and RMSE calculated using (10). The former is used to measure DE performance while the latter is used to measure the accuracy of the PDE solution approximation. The search histories of the five DEs for solving PDE1-6 are illustrated in Figures 5, 6, 7, 8, 9, and 10, respectively. From Table 1 based on the fitness value, it is shown that the proposed DE with population variation gives the best optimisation results for all test cases with only DE/ran/1/exp being comparable in cases of PDE4-6. The worst performer is DE/current-to-best/2/exp. From the search histories, it can be seen that DE/current-to-best/2/exp always gets struck at a local minimum and its search histories for PDE5-6 are discarded from the figures. For PDE5-6, all DEs cannot approach to zero. This could possibly be due to unevenly distributed nodes as shown in Figures 3 and 4.

For approximation accuracy based on RMSE, the results from the literature [24] are compared. It should be noted that, in [24], different fitness function is used, thus, optimisation convergence from [24] and this work cannot be compared. Also, the nodal distributions of PDE5-6 from [24] are also different from those used in this work. From the table, DE-New gives the best results for PDE1-3 while the approach in [24] gives the best results in cases of PDE4-6. Nevertheless, it should be noted that the previous work used ES in combination with a partial sum Fourier cosine series with approximately 1,000,000 function evaluations which is twice the total number of function evaluations used in this paper. DE-New significantly outperforms all other DE versions for PDE1-3. It is said to be as good as DE/ran/1/exp in cases of PDE4 while DE/ran/1/exp along with DE/ran/1/bin is slightly better for the last two test cases.

Graphical comparisons of the approximate solutions from using DE-New and the exact solutions of PDE1-6 are displayed in Figures 11, 12, 13, 14, 15, and 16, respectively. The left-hand side is an approximate solution while the right-hand side is the exact solution. It is shown that DE-New gives good approximation for PDE1-4 while the approximation of PDE5-6 is not so good. One of the notable reasons is that PDE5-6 have nodal distribution with poor quality. Figure 15 (PDE5) shows that the number of nodes on the boundary is insufficient. A similar conclusion can be said for PDE6 (Figure 16).

(a)

(b)

(a)

(b)

(a)

(b)

(a)

(b)

(a)

(b)

(a)

(b)

5. Conclusions and Future Work

An alternative numerical meshless approach for solving PDEs based on evolutionary computation is proposed. A PDE problem is converted into an optimisation problem while, in this paper, DE is used as an optimiser. A new DE with a population variation concept is proposed. The results show that the proposed method in this paper is comparable to the previous work while using approximately twice less the number of function evaluations in the optimisation process. Most of the approximate solutions of PDE are accurate except for some cases which use poorly distributed nodes. The performance of an optimiser is probably the most important factor for this type of meshless analysis. The choice of approximate function also affects the accuracy of estimating the PDE solutions. Apart from that, nodal distribution also plays a vital role. The weak points can be listed as difficulty in dealing with equality constraints of a metaheuristic, and its slow convergence rate. Moreover, the calculation of WRF is, to some extent, difficult in the sense that adjacent nodes for each subarea have to be identified initially.

For our future work, the performance enhancement of EAs or MHs is the first issue to be dealt with. EAs for optimisation with equality constraints should be rigorously investigated. The hybrid between EAs and some efficient gradient-based optimisers will be first focused. The choice of approximate functions will be investigated. Applications to real-world problems will be demonstrated.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

References

E. Onate, J. Rojek, M. Chiumenti, S. R. Idelsohn, F. D. Pin, and R. Aubry, “Advances in stabilized finite element and particle methods for bulk forming processes,” Computer Methods in Applied Mechanics and Engineering, vol. 195, no. 48-49, pp. 6750–6777, 2006.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
B. T. Tang, Z. Zhao, H. Hagenah, and X. Y. Lu, “Energy based algorithms to solve initial solution in one-step finite element method of sheet metal stamping,” Computer Methods in Applied Mechanics and Engineering, vol. 196, no. 17–20, pp. 2187–2196, 2007.
View at: Publisher Site | Google Scholar
J. Nam, M. Behr, and M. Pasquali, “Space-time least-squares finite element method for convection-reaction system with transformed variables,” Computer Methods in Applied Mechanics and Engineering, vol. 200, no. 33–36, pp. 2562–2576, 2011.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
S. R. Idelsohn, E. Onate, F. D. Pin, and N. Calvo, “Fluid-structure interaction using the particle finite element method,” Computer Methods in Applied Mechanics and Engineering, vol. 195, no. 17-18, pp. 2100–2123, 2006.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
H. P. Chu and C. L. Chen, “Hybrid differential transform and finite difference method to solve the nonlinear heat conduction problem,” Communications in Nonlinear Science and Numerical Simulation, vol. 13, no. 8, pp. 1605–1614, 2008.
View at: Publisher Site | Google Scholar
J. Zhao and R. M. Corless, “Compact finite difference method for integro-differential equations,” Applied Mathematics and Computation, vol. 177, no. 1, pp. 271–288, 2006.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
I. Bijelonja, I. Demirdzic, and S. Muzaferija, “A finite volume method for incompressible linear elasticity,” Computer Methods in Applied Mechanics and Engineering, vol. 195, no. 44–47, pp. 6378–6390, 2006.
View at: Publisher Site | Google Scholar
X. Nogueira, L. C. Felgueroso, I. Colominas, and H. Gomez, “Implicit large eddy simulation of non-wall-bounded turbulent flows based on the multiscale properties of a high-order finite volume method,” Computer Methods in Applied Mechanics and Engineering, vol. 199, no. 9–12, pp. 615–624, 2010.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
A. G. Malan and O. F. Oxtoby, “An accelerated, fully-coupled, parallel 3D hybrid finite-volume fluid-structure interaction scheme,” Computer Methods in Applied Mechanics and Engineering, vol. 253, pp. 426–438, 2013.
View at: Publisher Site | Google Scholar | MathSciNet
L. B. Lucy, “A numerical approach to the testing of the fission hypothesis,” The Astronomical Journal, vol. 82, pp. 1013–1024, 1977.
View at: Publisher Site | Google Scholar
R. A. Gingold and J. J. Monaghan, “Smoothed particle hydrodynamics: theory and application to non-spherical stars,” Monthly Notices of the Royal Astronomical Society, vol. 181, pp. 375–389, 1977.
View at: Publisher Site | Google Scholar
T. Belytschko, Y. Y. Lu, and L. Gu, “Element-free Galerkin methods,” International Journal for Numerical Methods in Engineering, vol. 37, no. 2, pp. 229–256, 1994.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
W. K. Liu, S. Jun, and Y. F. Zhang, “Reproducing Kernel particle methods,” International Journal for Numerical Methods in Fluids, vol. 20, no. 8-9, pp. 1081–1106, 1995.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
S. N. Atluri and S. Shen, “The Meshless Local Petrov-Galerkin (MLPG) method: a simple & less-costly alternative to the finite element and boundary element methods,” Computer Modeling in Engineering & Sciences, vol. 3, no. 1, pp. 11–51, 2002.
View at: Google Scholar | Zentralblatt MATH | MathSciNet
S. N. Atluri and T. Zhu, “A new Meshless Local Petrov-Galerkin (MLPG) approach in computational mechanics,” Computational Mechanics, vol. 22, no. 2, pp. 117–127, 1998.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
S. N. Atluri and T. Zhu, “The Meshless Local Petrov-Galerkin (MLPG) approach for solving problems in elasto-statics,” Computational Mechanics, vol. 25, no. 2, pp. 169–179, 2000.
View at: Google Scholar
K. M. Liew, X. Zhao, and A. J. M. Ferreira, “A review of meshless methods for laminated and functionally graded plates and shells,” Composite Structures, vol. 93, no. 8, pp. 2031–2041, 2011.
View at: Publisher Site | Google Scholar
V. P. Nguyen, T. Rabczuk, St. Bordas, and M. Duflot, “Meshless methods: a review and computer implementation aspects,” Mathematics and Computers in Simulation, vol. 79, no. 3, pp. 763–813, 2008.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
S. Wang, S. Li, Q. Huang, and K. Li, “An improved collocation meshless method based on the variable shaped radial basis function for the solution of the interior acoustic problems,” Mathematical Problems in Engineering, vol. 2012, Article ID 632072, 20 pages, 2012.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
S. Lyu, C. Wu, and S. Zhang, “Application of the RBF method to the estimation of temperature on the external surface in laminar pipe flow,” Mathematical Problems in Engineering, vol. 2013, Article ID 205169, 11 pages, 2013.
View at: Publisher Site | Google Scholar | MathSciNet
S. F. Tu and C. S. Hsu, “Constructing HVS-based optimal substitution matrix using enhanced differential evolution,” Mathematical Problems in Engineering, vol. 2013, Article ID 824239, 10 pages, 2013.
View at: Publisher Site | Google Scholar
S. L. Mei, “HAM-based adaptive multiscale meshless method for burgers equation,” Journal of Applied Mathematics, vol. 2013, Article ID 248246, 10 pages, 2013.
View at: Publisher Site | Google Scholar | MathSciNet
M. Babaei, “A general approach to approximate solutions of nonlinear differential equations using particle swarm optimization,” Applied Soft Computing, vol. 13, no. 7, pp. 3354–3365, 2013.
View at: Publisher Site | Google Scholar
J. M. Chaquet and E. J. Carmona, “Solving differential equations with Fourier series and evolution strategies,” Applied Soft Computing, vol. 12, no. 9, pp. 3051–3062, 2012.
View at: Publisher Site | Google Scholar
R. Storn and K. Price, “Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces,” Tech. Rep. TR-95-012, International Computer Science Institute, 1995.
View at: Google Scholar
X. Wang and S. Zhao, “Differential evolution algorithm with self-adaptive population resizing mechanism,” Mathematical Problems in Engineering, vol. 2013, Article ID 419372, 14 pages, 2013.
View at: Publisher Site | Google Scholar
Z. Yang, K. Tang, and X. Yao, “Self-adaptive differential evolution with neighborhood search,” in Proceedings of the IEEE Congress on Evolutionary Computation (CEC '08), pp. 1110–1116, Hong Kong, June 2008.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2014 Natee Panagant and Sujin Bureerat. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

5278

Downloads

1882

Citations