Research Article | Open Access
A New Stochastic Technique for Painlevé Equation-I Using Neural Network Optimized with Swarm Intelligence
A methodology for solution of Painlevé equation-I is presented using computational intelligence technique based on neural networks and particle swarm optimization hybridized with active set algorithm. The mathematical model of the equation is developed with the help of linear combination of feed-forward artificial neural networks that define the unsupervised error of the model. This error is minimized subject to the availability of appropriate weights of the networks. The learning of the weights is carried out using particle swarm optimization algorithm used as a tool for viable global search method, hybridized with active set algorithm for rapid local convergence. The accuracy, convergence rate, and computational complexity of the scheme are analyzed based on large number of independents runs and their comprehensive statistical analysis. The comparative studies of the results obtained are made with MATHEMATICA solutions, as well as, with variational iteration method and homotopy perturbation method.
The history of Painlevé equations is more than one century old. These are six second-order nonlinear irreducible equations that define new transcendental functions known as Painlevé Transcendents. These functions describe different physical processes and have been extensively used in both pure  and applied mathematics , along with theoretical physics . For instance, Painlevé Transcendents are used in solutions to Korteweg-de Vries (KdV), cylindrical KdV and Bussinesq equations [4, 5], bifurcations in nonlinear non-integral models , matrix models of quantum gravity with continuous limits [7, 8], among others. The history, importance, and applications of Painlevé transcendents can be seen elsewhere [9, 10]. In recent publications, Painlevé equation-I (PE-I) is used in Tronquée  and hyperasymptotic  solutions, as well as in modeling the viscous shocks in Hele-shaw flow and Stokes phenomena . Beside this, many researchers have employed PE-I in diverse fields of applied science and engineering [14–22].
In this article, our investigation is confined to find out solution of initial value problem (IVP) of nonlinear second-order PE-I, written in the following form:
In the present study, the strength of feed forward artificial neural networks (ANNs) is exploited for approximate mathematical model of PE-I. The real strength of such model to solve the differential equations can be achieved by using modern stochastic solvers for optimization of weights based on particle swarm optimization (PSO) technique hybrid with local search methods. For example, the solution of linear and nonlinear differential equations and their systems is reported extensively by many researchers [23–26]. Recently, solution of well-known fractional-order systems in engineering based on Bagley-Torvik and Riccati equations are other illustrative examples of such approaches [27, 28].
In this paper, ANNs supported with PSO and active set algorithm (ASA) are used to find the solution of IVP of PE-I. Monte-Carlo simulations of proposed scheme are performed and analyzed statistically to verify and validate its effectiveness. Comparison with standard MATHEMATICA, as well as homotopy perturbation method (HPM) [29, 30] and variational iteration method (VIM) [29, 30] results is also carried out for the given scheme.
The organization of the paper is as follows: the objective function creation with neural network mathematical modeling and its training methodology are introduced in Section 2. A brief introduction to particle swarm optimization algorithm is revealed in Section 3. A detailed application of the designed method along with some discussion on the results is presented in Section 4. Comparative analysis of the results is presented in Section 5. Last section concludes the findings along with some directions for future research.
2. Mathematical Modeling
The Feed-forward ANNs is well known to be used as a universal function approximator for any continuous function along with its derivatives on a compact set. In the modeling of differential equation through ANN methodology, following continuous mapping is used for the solution , first-order derivative and second-order derivative , respectively, given as [31–34] where , , and are real-valued bounded adaptive parameters, is the number of neurons, and being the activation function taken as log sigmoid for hidden layers:
The linear combination of the networks represented by (2) can arbitrarily model the nonlinear PE-I. It is called differential equation neural network (DE-NN) of the equation and its architecture is shown in Figure 1.
The objective or fitness function for solving PE-I has been formulated by defining an unsupervised error, given as the sum of mean square errors: where is the iteration number, represents the total number of iterations, and the error term is related with the differential equation and is written as
Here, the interval is divided into number of steps with step size and and are DE-NN given in set of (2).
Similarly, the error term is due to initial conditions and it is given as It is quite clear that, with the availability of weights , , and in DE-NN model for which the values of functions and approach zero, the value of unsupervised error also approaches zero. Therefore, the solution of the PE-I is approximated by as given in (2).
3. Learning Methodology
The learning procedures are introduced here for finding the unknown weights of DE-NN networks representing PE-I using PSO and ASA algorithms.
The swarm intelligence techniques often referred to as PSO algorithm was first introduced by Kennedy and Eberhart . It is a kind of global optimization technique which was based on the model of social behavior of bird flocking and fish schooling. Its discrete and continuous versions have been developed and used in different optimization problems of applied science and engineering. Few examples include the application to mobile communications, sensor networks, inventory control, multiprocessor scheduling, controls, stock market prediction, and steganographic methods [36–39] and so forth.
In PSO algorithm, each single candidate solution to an optimization problem is taken as a particle in the search space. The exploration of a problem space in PSO algorithm is made with a population of particles called a swarm. All particles in the swarm have own fitness values associated with problem specific objective function. The PSO algorithm is initialized with a swarm of particles randomly and is used to search for optimal solution iteratively. The broader spread of initial swarm results in optimal performance of the algorithm. The position and the velocity of each particle are updated according to its known previous local best position and the global best position of all particles in the swarm so far. The updating formula for each particle velocity and position in continuous standard PSO is written as where is vector to represent th particle of the swarm, , is an integer giving the total number of particles in a swarm. The is the velocity vector associated with th particle, and are the local and global social acceleration constants, respectively, is the inertia weight linearly decreasing over the course of search between 0 and 1, and and are random vectors with elements distributed between 0 and 1.
The elements of velocity are bounded as , where is maximum velocity. If the velocity goes beyond its maximum value, it will be set to . This parameter has its own importance and controls the convergence rate of the algorithm. The execution of the algorithm is terminated on the criterion to maximum flights/cycles is completed or a specific value of the fitness is achieved. The generic flowchart showing the process of proposed algorithm is presented in Figure 2.
The algorithm is given in the following steps.
Step 1. Initialization. The initial swarm of particle is generated randomly using bounded real values. Each particle represents the candidate solution with as many members as number of unknown parameters in DE-NN networks. The scattering in values of initial swarm is maintained for better search space for the algorithm. Initialize the values of parameters as given in Table 1 before execution of algorithm.
Step 3. Termination Criteria. Terminate the algorithm if any of following criteria satisfies the following:(i)predefined value of fitness achieved that is, ,(ii)number of maximum flights/cycles is executed.
If yes, then go to Step 6.
Step 4. Ranking. Rank each particle on the basis of minimum of the fitness function values.
Step 6. Refinement. MATLAB built-in formulation is used for running active set algorithm (ASA) for fine tuning of the results. The global best particle of PSO is used as a start point of the algorithm and other parameter settings for ASA is also provided in Table 1.
Step 7. Statistical Analysis. Store the value of global best particle along with its fitness value and time of executions for algorithm. Repeat Step 1 to 6 for sufficient large number of runs for reliable statistical analysis.
4. Simulation and Results
The well-known analytic solvers like adomian decomposition method (ADM), variational iteration method (VIM), homotopy perturbation method (HPM), homotopy analysis method (HAM), and their modified versions are used extensively to provide the solutions of strong nonlinear Painlevé equations [29, 30, 40–42]. The generic solution is provided in all these mentioned methods in the form of infinite series, but the results are determined using finite number of terms, whose accuracy can be enhanced with the increase in number of terms.
Before describing the designed approach, a brief description of VIM and HPM has been provided for PE-I. The approximate solution of for PE-I (1) in case of both VIM and HPM is represented in the form of a function . To describe the VIM, consider the following nonlinear differential equation: where and are linear and nonlinear operators, respectively, and is an inhomogeneous term. In standard VIM, the correct functional can be constructed as here, represents a general Lagrangian multiplier, and are the present and next approximation, ũ represents restricted variation, that is, . With suitable choice of , the fixed point of the correction functional (9) can be considered as an approximate solution of the equation and successive approximation of the solution will be readily obtained upon by determining the Lagrange multiplier. Consequently, the solution is given by . For interesting readers, the detail description of VIM along with its applicability to various kinds of differential equations can be seen in [43–46].
The governing iterative formulation for solving the PE-I using VIM with the optimal value for can be written as
The approximate solution by VIM for first three iterations using can be written as
On the other hand to explain the HPM, let us consider the following general equation of type: where represents the differential operator, is the domain and is a known analytical function. The operator can consist of linear part and a nonlinear part . In standard HPM, the Hopotopy can be constructed which satisfies here, is an embedding parameter and is an initial approximation of (12). By using small embedding parameter , the th solution of (12) can be given as power series in , that is, for , the approximate solution for (12) is given as
The method provides enough convergence in most of the cases and the rate of convergence depends on the nonlinear operator . The detail description of the HPM and its applications can be seen in [47–50].
Inserting instead of and comparing the coefficients of 1, 2, 3, 4, and 5 powers of , the approximate solutions by HPM for PE-I are, respectively, given as
Along with the analytical solutions given by VIM and HPM, the results are also calculated with MATHEMATICA . These results are used for comparison with the solution provided by the proposed design scheme.
Mathematical model of the equation is made with DE-NN networks given by (2) by taking 10 numbers of neurons, resulting in 30 unknown weights in DE-NN networks. The weights are taken as bounded real numbers between −50 to 50. The initial swarm consists of 160 particles, each with 30 numbers of elements, which is equivalent to the number of weights for DE-NN networks. Input of the training set is taken as with a step of . The fitness function provided in (4) can be formulated as
The PSO algorithm runs iteratively in order to compute the minimum of fitness function (19) and the best particle of PSO technique is passed to ASA algorithm as a start point for rapid local search. The parameter settings for the execution of algorithms are provided in Table 1.
Similarly, hundred independent runs of PSO and PSO hybrid with ASA (PSO-ASA) algorithms have been performed for finding the optimal weights. The weights obtained by the algorithms are used in (2) to obtained the proposed solutions of PE-I. One set of weights for which the value of fitness function and for PSO and PSO-ASA, respectively, are provided in Table 2. Results calculated by these weights are provided in Tables 3 and 4 for inputs between 0 and 1 with a step of 0.1. The solutions of 3rd order VIM, 5th order HPM for the same inputs are also given in Table 3 while detailed information has been provided in Table 4. It can be seen that the value of mean absolute error (MAE) for 3rd order VIM, 5th order HPM, PSO, and PSO-ASA techniques are , , , , respectively.
The values of minimum (MIN), maximum (MAX), mean, and standard deviation (STD), for absolute error (AE) , are calculated on the basis hundred independent runs for PSO and PSO-ASA methods. Results are provided in the Table 5 for inputs 0 and 1 with step of 0.1 for PE-I. It can be seen that values of AE on the basis of MIN are of the order 10−03 to 10−04 and 10−06 to 10−08 for PSO and PSO-ASA algorithms, while the values of AE based on mean lies in the range 10−01 to 10−02 and 10−04 to 10−05, respectively, for said solvers. The lowest values of statistical parameters are found by hybrid technique PSO-ASA than that of PSO algorithm.
The accuracy and convergence of the algorithms are analyzed further by calculating the values of MAE and fitness achieved (FA) for 100 independents runs. Results are shown graphically on semilog scale in order to elaborate the difference in the values. The value of FA and MAE against number of independent runs are shown graphically in Figure 3(a) for PSO and PSO-ASA methods. The numbers of independent runs are arranged on the basis of ascending order of the fitness achieved, as well as values of MAE and are plotted in Figure 3(b). On the basis of acceptability criteria of , the convergence for PSO and PSO-ASA is 1% and 100%, respectively.
5. Comparative Analysis of Results
In this section, the comparative study for results is presented based on values of MAE, execution time (ET), mean fitness achieved (MFA), global mean absolute error (GMAE), number of grid points, and convergence rate for PSO and PSO-ASA proposed solvers.
The effects on results by changing the number of gird points used in fitness function (19) are analyzed. By taking the number of grid points 21 instead of 11, the new fitness function is formulated using the input of the training set with a step of and given as
Hundred independents runs of PSO and PSO-ASA method are executed for finding the appropriate weights and subsequently the solution of IVP of PE-1. The MFA is defined as average values of fitness achieved taken over all independent runs of the algorithm. Similarly, GMAE is average value of mean absolute error for all independents runs of solvers. The values of MFA and GMAE are calculated for PSO and PSO-ASA techniques using both 11 and 21 grids points and results are tabulated in Table 6. It is found that the values of MFA are of the order 10−02 and 10−06 for PSO and PSO-ASA, respectively. The values of MAE are matched with MATHEMATICA solution upto 2 to 3 and 5 to 8 decimal places for PSO and PSO-ASA techniques. The value of GMAE for 11 and 21 grids points are determined and given in Table 6. It is found that values of GMAE are of the order 10−04 for both the cases for inputs between 0 and 1 with a step of . So, by increasing number of grid points the same level of the accuracy in results is maintained for both PSO and PSO-ASA techniques. The increase in number of grids points, that is, decrease in the mesh size , does not guaranty improvement of results for stochastic solvers and same is observed in our simulations by changing the value of step size to .
The convergence rates of the proposed stochastic solvers are determined based on the value of MAE , 10−03, and 10−04, as well as the value of MFA , 10−04, and 10−05. Results are tabulated in Table 6 for both PSO and PSO-ASA techniques. It can be seen that, with the acceptability of the results based on value of MFA , the hybrid approach PSO-ASA achieved this criteria for 99% and 98% of the independents runs for 11 and 21 numbers of grid points, respectively. In addition to that, results are consistently convergent for larger number of independents runs for PSO-ASA method than that of PSO algorithms.
Moreover, the computational complexity is analyzed by calculating the time of execution of each algorithm for 100 numbers of independent runs for both and . The values for minimum execution time (MIN-ET), maximum execution time (MAX-ET), mean execution time (M-ET), and standard deviation of execution time (STD-ET) are provided in the Table 6. It can be seen that values of M-ET for PSO and PSO-ASA techniques are 76.93 seconds and 92.17 seconds, respectively, for , and almost double values of M-ET in case for . The time taken for the execution for hybrid approach PSO-ASA is relatively greater than that of PSO algorithm, but this can be overshadow by its invariable dominance in accuracy and convergence. The time analysis provided in this article is carried out using Dell Latitude D630 laptop with Intel(R) Core(TM) 2 Duo CPU T9300, 2.50 GHz, and running with MATLAB version R2009b.
On the basis of the simulations and their comparative studies provided in the last sections, it can be concluded that the IVP of PE-I can be solved effectively by the designed computational intelligence algorithms using neural networks supported with PSO and ASA techniques. Results obtained using the weights for DE-NN networks of PE-I trained by PSO-ASA algorithm are found to be better than from PSO techniques. The values of absolute error from MATHEMATICA solution of PSO-ASA lies in the range 10−04 to 10−07 and match upto 5 to 7 decimal places with state of art analytical techniques of VIM and HPM.
On the basis of 100 independent runs for each algorithm to solve PE-I using 11 and 21 grid points, the PSO-ASA technique is found to be invariable superior than PSO algorithm using the criteria based on the values of mean absolute error, global mean absolute error, mean fitness achieved, and convergence rate. However, as far as computational complexity is concerned that PSO-ASA technique is slightly more computational exhaustive, but it can be overlook due to its overall dominance in the performance. This study can also be extended to use other biological inspired computational algorithms for solving the PE-I, as well as other Painlevé transcendents.
The authors gratefully acknowledge Professor Dr. Adiqa Kausar Kiani for providing her valuable suggestions to improve the presentation of the paper.
- A. P. Bassom, P. A. Clarkson, and A. C. Hicks, “Numerical studies of the fourth Painlevé equation,” IMA Journal of Applied Mathematics, vol. 50, no. 2, pp. 167–139, 1993.
- J. He, “Variational iteration method—a kind of non-linear analytical technique: some examples,” International Journal of Non-Linear Mechanics, vol. 34, no. 4, pp. 699–708, 1999.
- J. H. He, “Some asymptotic methods for strongly nonlinear equations,” International Journal of Modern Physics B, vol. 20, no. 10, pp. 1141–1199, 2006.
- M. J. Ablowitz and H. Segur, Solitons and the Inverse Scattering Transform, Studies in Applied Mathematics 4, SIAM, Philadelphia, Pa, USA, 1981.
- M. Tajiri and S. Kawamoto, “Reduction of KdV and cylindrical KdV equations to Painlevé equation,” Journal of the Physical Society of Japan, vol. 51, no. 5, pp. 1678–1681, 1982.
- R. Haberman, “Slowly varying jump and transition phenomena associated with algebraic bifurcation problems,” SIAM Journal on Applied Mathematics, vol. 37, no. 1, pp. 69–106, 1979.
- A. S. Fokas, A. R. Its, and A. V. Kitaev, “Discrete Painlevé equations and their appearance in quantum gravity,” Communications in Mathematical Physics, vol. 142, no. 2, pp. 313–344, 1991.
- A. S. Fokas, A. R. Its, and A. V. Kitaev, “Matrix models of two-dimensional quantum gravity and isomonodromic solutions of “Discrete Painlevé”equations,” Journal of Mathematical Sciences, vol. 73, no. 4, pp. 415–429, 1995.
- M. J. Ablowitz and P. A. Clarkson, Solitons, Nonlinear Evolution Equation and Inverse Scattering, Cambridge University Press, 1991.
- V. I. Gromak, I. Laine, and S. Shimomura, Painleve Differential Equations in the Complex Plane, Walter de Gruyter, New York, NY, USA, 2002.
- D. Dai and L. Zhang, “On tronquée solutions of the first Painlevé hierarchy,” Journal of Mathematical Analysis and Applications, vol. 368, no. 2, pp. 393–399, 2010.
- A. B. O. Daalhuis, “Hyperasymptotics for nonlinear odes ii. The first Painlevé equation and a second-order riccati equation,” Proceedings of the Royal Society A, vol. 461, no. 2062, pp. 3005–3021, 2005.