Research Article  Open Access
Jürgen Geiser, "Modified Jacobian Newton Iterative Method: Theory and Applications", Mathematical Problems in Engineering, vol. 2009, Article ID 307298, 24 pages, 2009. https://doi.org/10.1155/2009/307298
Modified Jacobian Newton Iterative Method: Theory and Applications
Abstract
This article proposes a new approach to the construction of a linearization method based on the iterative operatorsplitting method for nonlinear differential equations. The convergence properties of such a method are studied. The main features of the proposed idea are the linearization of nonlinear equations and the application of iterative splitting methods. We present an iterative operatorsplitting method with embedded Newton methods to solve nonlinearity. We confirm with numerical applications the effectiveness of the proposed iterative operatorsplitting method in comparison with the classical Newton methods. We provide improved results and convergence rates.
1. Introduction
In this paper we propose a modified JacobianNewton iterative method to solve nonlinear differential equations. In the first paper we concentrate on ordinary differential equations, but numerical results are also obtained for partial differential equations. Basic studies of the operatorsplitting methods are found in [1, 2]. Further important research was carried out to obtain a higher order for the splitting methods (see [3]). For this reason, the iterative splitting methods became more important for linear and nonlinear differential equations, while simple increasing of iteration steps affects the order of the scheme (see [4]). An interesting topic is Newton's methods for nonlinear problems with specifications for numerical implementations, (see [5]). Efficient modifications of Newton's methods, with regard to the computation of the Jacobian matrices are discussed in [6]. In our paper we discuss the benefit of the combination of splitting and linearization methods (see theoretical frameworks [7, 8]).
The outline of the paper is as follows. For our mathematical model we describe the convectiondiffusionreaction equation in Section 2. The fractional splitting is introduced in Section 3. We present the iterative splitting methods in Section 4. Section 5 discusses the Newton methods and their modifications. In Section 6 we present the numerical results from the solution of selected model problems. We end the article in Section 7 with a conclusion and comments.
2. Mathematical Model
The motivation for the study presented below originates from a computational simulation of heattransfer [9] and convectiondiffusionreactionequations [10–13].
In the present paper we concentrate on ordinary differential equations, given as where the initial condition is . The operators and can be spatially discretized operators, that is, they can correspond to the discretized in space convection and diffusion operators (matrices). In the following, we deal with bounded nonlinear operators.
The aim of this paper is to present a new method based on Newton and iterative schemes.
In the next section we discuss the decoupling of the timescale with a firstorder fractional splitting method.
3. FractionalSplitting Methods of FirstOrder for Linear Equations
First we describe the simplest operatorsplitting, which is called sequential operatorsplitting, for the following linear system of ordinary differential equations:where the initial condition is . The operators and are linear and bounded operators in a Banach space (see also Section 2).
The sequential operatorsplitting method is introduced as a method that solves two subproblems sequentially, where the different subproblems are connected via the initial conditions. This means that we replace the original problem (3.1) with the subproblems where the splitting timestep is defined as . The approximated solution is .
Clearly, the replacement of the original problem with the subproblems usually results in an error, called splitting error. The splitting error of the sequential operatorsplitting method can be derived as (cf., e.g., [1, 2]). where is the commutator of and . Consequently, the splitting error is when the operators and do not commute, otherwise the method is exact. Hence, by definition, the sequential operatorsplitting is called the firstorder splitting method.
4. The Iterative Splitting Method
The following algorithm is based on the iteration with fixed splitting discretization stepsize . On the time interval we solve the following subproblems consecutively for : where is the known split approximation at the time level (see [14]).
Remark 4.1. We can generalize the iterative splitting method to a multiiterative splitting method by introducing new splitting operators, for example, spatial operators. Then we obtain multiindices to control the splitting process; each iterative splitting method can be solved independently, while connecting with further steps to the multisplitting methods. In the following we introduce the multiiterative splitting method for a combined timespace splitting method.
5. The Modified JacobianNewton Methods and FixpointIteration Methods
In this section we describe the modified JacobianNewton methods and Fixpointiteration methods.
We propose for weak nonlinearities, for example, quadratic nonlinearity, the fixpoint iteration method, where our iterative operator splitting method is one, see [4]. For stronger nonlinearities, for example, cubic or higher order polynomial nonlinearities, the modified Jacobian method with embedded iterativesplitting methods is suggested.
The point of embedding the splitting methods into the Newton methods is to decouple the equation systems into simpler equations. Such simple equation systems can be solved with scalar Newton methods.
5.1. The Altered JacobianNewton Iterative Methods with Embedded Sequential Splitting Methods
We confine our attention to timedependent partial differential equations of the formwhere are linear and densely defined in the real Banach space , involving only spatial derivatives of , see [8]. We assume also that we have a weak nonlinear operator with and , where and are constant factors.
In the following we discuss the embedding of a sequential splitting method into the Newton method.
The altered JacobianNewton iterative method with an embedded iterative splitting method is given as follows.
Newton's Method
and we can compute ,
where is the Jacobian matrix and
We stop the iterations when we obtain: , where is an error bound, for example, .
We assume the spatial discretization, with spatial grid points, and obtain the differential equation system:where and is the number of spatial grid points.
The Jacobian matrix for the equation system is given as:where .
The modified Jacobian is given as:where .
By embedding the sequential splitting method we obtain the following algorithm. We decouple into two equation systems:where the results of the methods are , and , .
Thus we have to solve two Newton methods, each in one equations system. The contribution is to reduce the Jacobian matrix into a diagonal entry, for example, with a weighted Newton method, see [15]. The splitting method with embedded Newton method is given as follows.
Algorithm 5.1. We assume the spatial operators and are discretized, for example, finite difference or
finite element methods; further all initial conditions and boundary conditions
are discrete given. Then we can apply the Newton's method in its discrete form
as:where and are the iteration indices, and the maximal iterative steps for each part of
the Newton's method. The maximal iterative steps allow us to have at least an
error of:
where is the error bound, for example, .
The approximated solution is given as:
For the improvement method, we can apply the weighted Newton method. We try to skip the delicate outer diagonals in the Jacobian matrix and apply:where the function can be applied as a scalar, for example, , and the same with . It is important to ensure that is small enough to preserve the convergence.
Remark 5.2. If we assume that we discretize (5.5) with the backwardEuler method, for example, then we obtain the derivations and
We can apply the equation (5.9) analogously .
5.2. Iterative OperatorSplitting Method as a Fixpoint Scheme
The iterative operatorsplitting method is used as a fixpoint scheme to linearize the nonlinear operators, see [4, 16].
We confine our attention to timedependent partial differential equations of the form:where are linear and densely defined in the real Banach space , involving only spatial derivatives of , see [8]. In the following we discuss the standard iterative operatorsplitting methods as a fixpoint iteration method to linearize the operators.
We split our nonlinear differential equation (5.12) by applying:where the timestep is . The iterations are . is the starting solution, where we assume the solution is near , or . So we have to solve the local fixpoint problem. is the known split approximation at the time level .
The split approximation at time level is defined as . We assume the operators to be linear and densely defined on the real Banach space , for .
Here the linearization is done with respect to the iterations, such that are at least nondependent operators in the iterative equations, and we can apply the linear theory.
The linearization is at least in the first equation , and in the second equation .
We have with sufficient iterations (5.14).
RemarkThe linearization with the fixpoint scheme can be used for smooth or weak nonlinear operators, otherwise we lose the convergence behavior, while we did not converge to the local fixpoint, see [].
The second idea is based on the Newton method.
5.3. JacobianNewton Iterative Method with Embedded OperatorSplitting Method
The Newton method is used to solve the nonlinear parts of the iterative operatorsplitting method (see the linearization techniques in [, ]).
Newton Method
The function is given as:
The iteration can be computed as:
where (5.14) is the Jacobian matrix and (5.14)
and (5.14) is the solution vector of the spatial
discretized nonlinear equation.
We then have to apply the iterative operatorsplitting method and obtain:where the timestep is . The iterations are . is the starting solution and is the known split approximation at the timelevel . The results of the methods are .
Thus we have to solve two Newton methods and the contribution will be to reduce the Jacobian matrix, for example, skip the diagonal entries. The splitting method with the embedded Newton method is given as:where the timestep is . The iterations are: . is the starting solution and is the known split approximation at the timelevel . The results of the methods are .
For the improvement by skipping the delicate outer diagonals in the Jacobian matrix, we apply , and analogously .
Remark 5.4. For the iterative operatorsplitting method with the Newton iteration we have two iteration procedures. The first iteration is the Newton method to compute the solution of the nonlinear equations, and the second iteration is the iterative splitting method, which computes the resulting solution of the coupled equation systems. The embedded method is used for strong nonlinearities.
6. Numerical Results
In this section, we present the numerical results for nonlinear differential equation using several variations of the proposed Newton and iterative schemes as solvers.
6.1. First Numerical Example: Bernoulli Equation
As a nonlinear differential example, we choose the Bernoulli equation:where the analytical solution can be derived as (see also [16]):
Using we find that , so
We choose , , , , and, for example, .
The analytical solutions can be given as:
We divide the time interval , with , in intervals with length .
(1) The sequential operatorsplitting method with analytical solutions is given as follows.
We apply the quasilinear iterative operatorsplitting method: with the nonlinear operators . The result is given as .
We apply the Newton method and discretize the operators with time discretization methods such as backwardEuler or higher RungeKutta methods.
The analytical result for each equation part is given as: where the result is given as .
We can apply the simpler equations and solve the sequential operatorsplitting method.
(2) The sequential operatorsplitting method with embedded Newton method is given as follows.
We apply the quasilinear iterative operatorsplitting method: with the nonlinear operators . The result is given as .
We apply the Newton method and discretize the operators with time discretization methods such as backwardEuler or higher RungeKutta methods.
The splitting method with embedded Newton's method is given as where we discretize the equations and obtain the discretized operators:aswhere we have the initialization of the Newton's method as or .
For the second iteration equation we haveaswhere we have the initialization of the Newton's method as or .
The derivations are given as:
(3) The standard iterative operatorsplitting method is given as follows.
We apply the quasilinear iterative operatorsplitting method: with the nonlinear operators . The initialization of the fixpoint iteration is or with and .
For the iterations we can apply the analytical solution of each equation:
Further the iterative steps can be done.
(4) The Newton iterative method with embedded iterative operatorsplitting method is given as follows.
We apply the quasilinear iterative operatorsplitting method:with the nonlinear operators . The initialization of the fixpoint iteration is or .
The discretization of the nonlinear ordinary differential equation is performed with higherorder RungeKutta methods.
The Newton method is applied as:Here the timestep is . The iterations are . is the starting solution and is the known split approximation at the timelevel . The results of the methods are .
We apply the discretization methods for the iteration steps.
We discretize the equationsaswhere we have the initialization of the Newton's method as or .
For the second iteration equation we have:aswhere we have the initialization of the Newton's method as or .
The derivations are given as:
Our numerical results for the different methods are presented in Tables 1, 2, 3, and 4. The errors of the methods are shown in Figures 1, 2, and 3. We chose different iteration steps and time partitions. The error between the analytical and numerical solution is shown with the supremum norm at time .




The experiments show the reduced errors for more iteration steps and more time partitions. Because of the timediscretization method for ODEs, we restrict the number of iteration steps to a maximum of five. If we restrict the error bound to , two iteration steps and five time partitions give the most effective combination.
6.2. Second Numerical Example: Mixed ConvectionDiffusion and Burgers Equation
We deal with a 2D example which is a mixture of a convectiondiffusion and Burgers equation. We can derive an analytical solution:where , , and is the viscosity.
The analytical solution is given aswhere we compute accordingly.
We split the convectiondiffusion and the Burgers equation. The operators are given as: hence
For the first equation we apply the nonlinear Algorithm 5.1 and obtain and we obtain linear operators, because is known from the previous timestep.
In the second equation we obtain by using Algorithm 5.1: and we have linear operators.
We deal with different viscosities as well as different stepsizes in time and space. We have the following results (see Tables 5 and 6).


Figure 4 presents the profile of the 2D linear and nonlinear convectiondiffusion equation.
(a)
(b)
Remark 6.1. In the examples, we deal with more iteration steps to obtain higherorder convergence results. In the first test we have four iterative steps but a smaller viscosity () such that we can reach at least a secondorder method. In the second test we use a high viscosity about and get the secondorder result with two iteration steps. Here we see the loss of differentiability, that becomes stiff equation parts. To obtain the same results, we have to increase the number of iteration steps. Therefore we can show an improvement of the convergence order with respect to the iteration steps.
6.3. Third Numerical Example: Momentum Equation (Molecular Flow)
We deal with an example of a momentum equation, that is used to model the viscous flow of a fluid.where is the solution and , , , and are the parameters and is the unit matrix.
The nonlinear function is the viscosity flow, and is a constant velocity.
We can derive the analytical solution with respect to the first two test examples with the functions: For the splitting method our operators are given as:
We deal first with the onedimensional case,where is the solution and , , , and are the parameters.
Then the operators are given as: For the iterative operatorsplitting as fixed point scheme, we have the following results (see Tables 7 and 9).

Figure 5 presents the profile of the 1D momentum equation.
(a)
(b)
We have the following results for the 2D case (see Tables 10, 11, and 12).
Figure 6 presents the profile of the 2D momentum equation.
(a)
(b)
(c)
(d)
For the Newton operatorsplitting method we obtain the following functional matrices for the onedimensional case:For the twodimensional case, we use:
Here, we do not need the linearization and apply the standard iterative splitting method.
We only linearize the first split step and therefore we can relax this step with the second linear split step. Therefore we obtain stable methods, see [15]. For the Newton iterative method, we have the following results, see Table 8.





Remark 6.2. In the more realistic examples of 1D and 2D momentum equations, we can also observe the stiffness problem, which we obtain with a more hyperbolic behavior. In the 1D experiments we deal with a more hyperbolic behavior and can obtain at least firstorder convergence with two iterative steps. In the 2D experiments we obtain nearly secondorder convergence results with two iterative steps, if we increase the parabolic behavior, for example, larger and values. For such methods, we have to balance the usage of the iterative steps with refinement in time and space with respect to the hyperbolicity of the equations. At least we can obtain a secondorder method with more than two iterative steps. Therefore the stiffness influences the number of iterative steps.
7. Conclusion and Discussion
We present decomposition methods for differential equations based on iterative and noniterative methods. The nonlinear equations are solved with embedded Newton's methods. We present new ideas on linearization to obtain more accurate results. The superiority of the new embedded Newton's methods over the traditional sequential methods is demonstrated in examples, especially through their simple implementation. Further, we have the smoothing properties of the iterative scheme that allow a balance between the nonlinear and the linear terms. The results show more accurate solutions with respect to time decomposition. In the future the iterative operatorsplitting method can be generalized for multidimensional problems and also for nonsmooth and nonlinear problems in time and space. In next paper we discuss error analysis of nonlinear methods.
References
 G. I. Marchuk, “Some applicatons of splittingup methods to the solution of problems in mathematical physics,” Aplikace Matematiky, vol. 1, pp. 103–132, 1968. View at: Google Scholar
 G. Strang, “On the construction and comparison of difference schemes,” SIAM Journal on Numerical Analysis, vol. 5, no. 3, pp. 506–517, 1968. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 H. Yoshida, “Construction of higher order symplectic integrators,” Physics Letters A, vol. 150, no. 5–7, pp. 262–268, 1990. View at: Publisher Site  Google Scholar  MathSciNet
 J. F. Kanney, C. T. Miller, and C. T. Kelley, “Convergence of iterative splitoperator approaches for approximating nonlinear reactive problems,” Advances in Water Resources, vol. 26, no. 3, pp. 247–261, 2003. View at: Publisher Site  Google Scholar
 P. Deuflhard, Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithm, vol. 35 of Springer Series in Computational Mathematics, Springer, Berlin, Germany, 2004. View at: Zentralblatt MATH  MathSciNet
 P. Deuflhard, “A modified Newton method for the solution of illconditioned systems of nonlinear equations with application to multiple shooting,” Numerische Mathematik, vol. 22, no. 4, pp. 289–315, 1974. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 M. Ruzicka, Nichtlineare Funktionalanalysis, Springer, Berlin, Germany, 2004. View at: Zentralblatt MATH
 E. Zeidler, Nonlinear Functional Analysis and Its Applications. II/B: Nonlinear Monotone Operators, Springer, New York, NY, USA, 1990. View at: Zentralblatt MATH  MathSciNet
 J. Geiser, O. Klein, and P. Philip, “Anisotropic thermal conductivity in apparatus insulation: numerical study of effects on the temperature field during sublimation growth of silicon carbide single crystals,” preprint no. 1034 of Weierstrass Institute for Applied Analysis and Stochastics, Berlin, Germany, 2005. View at: Google Scholar
 J. Geiser, “Discretization methods with embedded analytical solutions for convection dominated transport in porous media,” in Proceedings of the 3rd International Conference on Numerical Analysis and Its Applications (NAA '04), vol. 3401 of Lecture Notes in Computer Science, pp. 288–295, Springer, Rousse, Bulgaria, JuneJuly 2004. View at: Google Scholar
 W. Hundsdorfer and J. Verwer, Numerical Solution of TimeDependent AdvectionDiffusionReaction Equations, vol. 33 of Springer Series in Computational Mathematics, Springer, Berlin, Germany, 2003. View at: Zentralblatt MATH  MathSciNet
 K. H. Karlsen, K.A. Lie, J. R. Natvig, H. F. Nordhaug, and H. K. Dahle, “Operator splitting methods for systems of convectiondiffusion equations: nonlinear error mechanisms and correction strategies,” Journal of Computational Physics, vol. 173, no. 2, pp. 636–663, 2001. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 K. H. Karlsen and N. H. Risebro, “Corrected operator splitting for nonlinear parabolic equations,” SIAM Journal on Numerical Analysis, vol. 37, no. 3, pp. 980–1003, 2000. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 I. Faragó and J. Geiser, “Iterative operatorsplitting methods for linear problems,” International Journal of Computational Science and Engineering, vol. 3, no. 4, pp. 255–263, 2007. View at: Publisher Site  Google Scholar
 C. T. Kelly, Iterative Methods for Linear and Nonlinear Equations, Frontiers in Applied Mathematics, SIAM, Philadelphia, Pa, USA, 1995. View at: Zentralblatt MATH  MathSciNet
 J. Geiser, “Iterative operatorsplitting methods with higherorder time integration methods and applications for parabolic partial differential equations,” Journal of Computational and Applied Mathematics, vol. 217, no. 1, pp. 227–242, 2008. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 K. H. Karlsen and N. H. Risebro, “An operator splitting method for nonlinear convectiondiffusion equations,” Numerische Mathematik, vol. 77, no. 3, pp. 365–382, 1997. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
Copyright
Copyright © 2009 Jürgen Geiser. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.