Iterative Methods for Nonlinear Equations or Systems and Their ApplicationsView this Special Issue
Research Article | Open Access
On a Newton-Type Method for Differential-Algebraic Equations
This paper deals with the approximation of systems of differential-algebraic equations based on a certain error functional naturally associated with the system. In seeking to minimize the error, by using standard descent schemes, the procedure can never get stuck in local minima but will always and steadily decrease the error until getting to the solution sought. Starting with an initial approximation to the solution, we improve it by adding the solution of some associated linear problems, in such a way that the error is significantly decreased. Some numerical examples are presented to illustrate the main theoretical conclusions. We should mention that we have already explored, in some previous papers (Amat et al., in press, Amat and Pedregal, 2009, and Pedregal, 2010), this point of view for regular problems. However, the main hypotheses in these papers ask for some requirements that essentially rule out the application to singular problems. We are also preparing a much more ambitious perspective for the theoretical analysis of nonlinear DAEs based on this same approach.
Differential-algebraic equations are becoming increasingly important in a lot of technical areas. They are currently the standard modeling concept in many applications such as circuit simulation, multibody dynamics, and chemical process engineering; see for instance [1–5] with no attempt to be exhaustive.
A basic concept in the analysis of differential-algebraic equations is the index. The notion of index is used in the theory of DAEs for measuring the distance from a DAE to its related ODE. The higher the index of a DAE is, the more difficulties are for its numerical solution. There are different index definitions, but for simple problems they are identical. On more complicated nonlinear and fully implicit systems they can be different (see  and the references therein.)
For simplicity, we focus our attention on problems of the form where is a given, eventually singular, matrix depending on . More general situations can be allowed. This type of equations arises, for instance, in the functional analytic formulation of the initial value problem for the Stokes as well as for the linearized Navier-Stokes or Oseen equations .
For the approximation of these equations collocation-type methods are usually used. These methods are implicit, and we need to solve a nonlinear system of equations in each iteration using a Newton’s type method. Given different coefficients , , there is a (unique for sufficiently small) polynomial of collocation of degree less than or equal to such that The collocation methods are defined by an approximation and are equivalent to implicit RK methods of stages for the coefficients The coefficients play the role of the nodes of the quadrature formula, and the associated coefficients are analogous to the weights. From (1.4), we can find implicit RK methods called Gauss of order , Radau IA and Radau IIA of order , and Lobatto IIIA of order . Also we can consider perturbed collocation methods like Lobato IIIC. (See  for more details).
A number of convergence results have been derived for these methods introducing the so-called -convergence theory. In [7, 8] the authors extend the -convergence theory to be valid for a class of nonautonomous weakly nonlinear stiff systems, in particular, including the linear case. As pointed out by the same authors, it is not clear if it is possible to cover, in a satisfactory way, highly nonlinear stiff problems, that is, problems where also the nonlinear terms are affected by large parameters. Moreover, any result should assume that, in each step, the associated nonlinear system is well approximated . In particular, we should be able to start with a good initial guess for the iterative scheme. This might be very restrictive for many stiff problems.
On the other hand, iterative methods are the typical tool to solve nonlinear systems of equations. In these schemes we compute a sequence of approximations solving associated linear problems. In this paper, we would like to introduce a new variational approach for the treatment of DAEs where we linearize the original equations obtaining an iterative scheme. Our ideas are based on the analysis of a certain error functional of the form to be minimized among the absolutely continuous paths with . Note that if is finite for one such path , then automatically is square integrable. This error functional is associated, in a natural way, with the Cauchy problem (1.1). Indeed, the existence of solutions for (1.1) is equivalent to the existence of minimizers for with vanishing minimum value. This is elementary.
In this initial contribution, we want to concentrate on the approximation issue through this perspective. We will place ourselves under the appropriate hypotheses so that there are indeed solutions for (1.1), that is, there are minimizers for the error with vanishing minimum value. In addition, we would like to guarantee that the main ingredients for the iterative approximating scheme to work are valid. More explicitly, our approach for the numerical approximation of such problems relies on three main analytical hypotheses that we take for granted here. (1) The Cauchy problem (1.1) admits a unique solution for every feasible initial condition (the definition of feasible path should depend on the index of the equation). (2) The linearization around any feasible, absolutely continuous, path with , always has a unique solution, and moreover, for some constant depending on , , and its derivatives, (3)The only solution of the problem is , for every feasible, absolutely continuous, path with .Here the superscript indicates transpose.
These requirements depend on the index of the equation and on some regularity on the pair . They should be more restrictive for equations with high index. More details can be found, for example, in [9, Theorem 3.9], where the authors consider DAEs transferable into standard canonical form. More precise information is outside of the scope of this paper. In any case, the equations verifying our hypotheses are, in general, a subclass of all analytically solvable systems.
In addition to the basic facts just stated on existence and uniqueness of solutions for our problems, the analysis of the approximation scheme, based on a minimization of the error functional , requires one main basic assumption on the nonlinearity . It must be smooth, so that is continuous and globally Lipschitz with constant (). Moreover, the main result of this paper demands a further special property on the map for every positive and small , there is so that This regularity is somehow not surprising as our approach here is based on regularity and optimality. On the other hand, that regularity holds for most of the important problems in applications. It certainly does in all numerical tests performed in this work. Our goal here is placed on the fact that this optimization strategy may be utilized to set up approximation schemes based on the minimization of the error functional. Indeed, we provide a solid basis for this approximation procedure. One very important and appealing property of our approach states that typical minimization schemes like (steepest) descent methods will work fine as they can never get stuck in local minima and converge steadily to the solution of the problem, no matter what the initialization is.
We should mention that we have already explored, in some previous papers, this point of view. Since the initial contribution , we have also treated the reverse mechanism of using first discretization and then optimality . The perspective of going through optimality and then discretization has already been indicated and studied in , though only for the steepest descent method, and without going through any further analytical foundation for the numerical procedure. However, the main hypotheses in these papers ask for some requirements that essentially rule out the application to singular problems. We will however address shortly  a complete treatment of DAEs with no a priori assumptions on existence and uniqueness. Rather, we will be interested in showing existence and uniqueness from scratch by examining the fundamental properties of the error functional .
On the other hand, variational methods have been used also before in the context of ODEs. See [14, 15], where numerical integration algorithms for finite-dimensional mechanical systems that are based on discrete variational principles are proposed and studied. This is one approach to deriving and studying symplectic integrators. The starting point is Hamilton’s principle and its direct discretization. In those references, some fundamental numerical methods are presented from that variational viewpoint where the model plays a prominent role.
The rest of the paper is divided in three sections. In Section 2 we introduce our variational approach and develop a convergence analysis. Section 3 introduces the numerical procedure. We analyze some numerical results in Section 4. Finally, we present the main conclusions in Section 5.
2. A Main Descent Procedure
We start with a fundamental fact for our approach.
Proposition 2.1. Let be a critical point for the error . Then is the solution of the Cauchy problem (1.1).
Proof. The proof is elementary. Based on the smoothness and bounds assumed on the mapping , we conclude that if is a critical point for the error , then ought to be a solution of the problem The vector-valued map is then a solution of the linear, nondegenerate problem The only solution of this problem, by our initial conditions on uniqueness of linearizations, is , and so is the solution of our Cauchy problem.
On the other hand, suppose we start with an initial crude approximation to the solution of our basic problem (1.1). We could take for all or . We would like to improve this approximation in such a way that the error is significantly decreased. We have already pointed out that descent methods can never get stuck on anything but the solution of the problem, under global lipschitzianity hypotheses.
It is straightforward to find the Gâteaux derivative of at a given feasible in the direction with . Namely This expression suggests a main possibility to select from. Choose such that In this way, it is clear that , and so the (local) decrease of the error is of the size . Finding requires solving the above linear problem which is assumed to have a unique solution by our main hypotheses in the introduction. In some sense, this is like a Newton method with global convergence.
Suppose is a feasible path in the interval so that , is square integrable, for a fixed constant , all , and the quantity measures how far such is from being a solution of our problem.
Theorem 2.2. For sufficiently small, the iterative procedure , starting from the above feasible and defining as the solution of the linear problem converges strongly in to the solution of (1.1).
Proof. Choose and so that
for some constant (see the main hypotheses in Section 1). We then solve for as the solution of the nonautonomous linear problem
and pretend to update to in such a way that the error for be less than the error for the current iteration . Note that
where we have used the differential equation satisfied by and the definition of . By our assumption on above,
provided that . Since we know that is the solution of a certain linear problem, by the upper bound assumed in Section 1 on the size of these solutions,
Assume that we select so small that
and then for all . By (2.9), (2.10), and (2.11), we can write
If, in addition, we demand, by making smaller if necessary,
then . Moreover, for all ,
All these calculations form the basis of a typical induction argument, verifying
It is therefore clear that the sum converges strongly in to the solution of our initial Cauchy problem in a small interval .
Since the various ingredients of the problem do not depend on , we can proceed to have a global approximation in a big interval by successively performing this analysis in intervals of appropriate small size. For instance, we can always divide a global interval into a certain number of subintervals of small length () with according to (2.14).
3. Numerical Procedure
Since our optimization approach is really constructive, iterative numerical procedures are easily implementable.(1)Start with an initial approximation compatible with the initial conditions (e.g. ).(2)Assume we know the approximation in .(3)Compute its derivative .(4)Compute the auxiliar function as the numerical solution of the problem by making use of a numerical scheme for DAEs with dense output (like collocation methods). (5)Change to by using the update formula (6)Iterate (3), (4), and (5), until numerical convergence. In practice, we use the stopping criterium
In particular, one can implement, in a very easy way, this numerical procedure using a problem-solving environment like MATLAB .
4. Some Experiments
In this section, we approximate some problems well known in the literature for a different index [5, 17, 18]. High-order accuracy and stability are major areas of interest in this type of simulations. We do not perform an analysis of the convergence conditions imposed in the above section. We are only interested to test numerically our approach.
In our approach we only need to approximate, with at least order one, the associated linear system for , in order to obtain the convergence of our scheme (see Theorem 2.2). The stability can be ensured by the fact that we approximate a linear problem using specific implicit methods . This is not the case with a general nonlinear problem , where we need to approximate well (with a Newton-type iterative method) the nonlinear system related to the implicitness of the scheme (see the above section). This approximation should be a difficult task due to the local (nonglobal) convergence of any iterative scheme for nonlinear problems.
In this section, we consider the convergent Lobatto IIIC method  valid for indexes 1–3, in order to approximate the associated linear problem for in each iteration. This method can be considered as a perturbation collocation method. The final error depends only on the stopping criterium. In the following examples, we stop the algorithm when and plot the solution and the approximation given by our approach.(i)Index 1  The solution of this problem is .(ii)Index 2  where The solution of this problem is .(iii) Index 3  The solution of this problem is .
In Figures 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10, we compare the solution of the corresponding three problems with the approximations given by our approach. The results are very satisfactory in all cases, obtaining always the convergence to the true solution. In a first look, the exact and computed solutions are indistinguishable, since after convergence the error is smaller than the tolerance () used in the stopping criterium. A more systematic and careful analysis of the numerical possibilities of the method will be pursued in the future.
A new variational approach to the analysis and numerical implementation of regular ODEs has been recently introduced in [10, 20]. Because of its flexibility and simplicity, it can easily be extended to treat other types of ODEs like differential-algebraic equations (DAEs). This has been precisely the main motivation for this paper: to explore how well those ideas can be adapted to this framework. In particular, extending to this context some of the analytical results and performing various numerical tests that confirm that indeed the variational perspective is worth pursuing. One remarkable feature is that this point of view only requires to count on good numerical schemes for linear problems, and this is the reason why it fits so well in other scenarios. Because of the many good qualities of this viewpoint, it can be considered and implemented in essentially all fields where differential equations are relevant. There is, then, a long way to go.
S. Amat and M. J. Légaz supported by MINECO-FEDER MTM2010-17508 (Spain) and by 08662/PI/08 (Murcia). P. Pedregal supported by MINECO-FEDER MTM2010-19739 (Spain).
- K. E. Brenan, S. L. Campbell, and L. R. Petzold, Numerical Solution of Initial-Value Problems in Differential-Algebraic Equations, SIAM, Philadelphia, Pa, USA, 1996.
- K. E. Brenan and L. R. Petzold, “The numerical solution of higher index differential/algebraic equations by implicit methods,” SIAM Journal on Numerical Analysis, vol. 26, no. 4, pp. 976–996, 1989.
- S. L. Campbell and E. Griepentrog, “Solvability of general differential algebraic equations,” SIAM Journal on Scientific Computing, vol. 16, no. 2, pp. 257–270, 1995.
- E. Hairer and G. Wanner, Solving Ordinary Differential Equations II: Stiff and Differential Algebraic Problems, vol. 14, Springer, Berlin, Germany, 1991.
- R. Riaza, Differential-Algebraic Systems, World Scientific, Singapore, 2008.
- E. Emmrich and W. Mehrmann, Analysis of Operator DiFFerential-Algebraic Equations Arising Influid Dynamics. Part I. The Finite Dimensional Case, 2010.
- W. Auzinger, R. Frank, and G. Kirlinger, “An extension of -convergence for Runge-Kutta methods,” Applied Numerical Mathematics, vol. 9, no. 2, pp. 91–109, 1992.
- W. Auzinger, R. Frank, and G. Kirlinger, “Modern convergence theory for stiff initial value problems,” Journal of Computational and Applied Mathematics, vol. 45, no. 1-2, pp. 5–16, 1993.
- T. Berger and A. Ilchmann, “On the standard canonical form of time-varying linear DAEs,” Quarterly of Applied Mathematics. In press.
- S. Amat and P. Pedregal, “A variational approach to implicit ODEs and differential inclusions,” ESAIM. Control, Optimisation and Calculus of Variations, vol. 15, no. 1, pp. 139–148, 2009.
- S. Amat, D. J. López, and P. Pedregal, “An optimization approach for the numerical approximation of differential equations,” Optimization. In press.
- P. Pedregal, “A variational approach to dynamical systems, and its numerical simulation,” Numerical Functional Analysis and Optimization, vol. 31, no. 7, pp. 1532–2467, 2010.
- S. Amat and P. Pedregal, “A constructive existence theorem for non-linear DAEs through a variational strategy,” in preparation.
- A. Lew, J. E. Marsden, M. Ortiz, and M. West, “Variational time integrators,” International Journal for Numerical Methods in Engineering, vol. 60, no. 1, pp. 153–212, 2004.
- J. E. Marsden and M. West, “Discrete mechanics and variational integrators,” Acta Numerica, vol. 10, pp. 357–514, 2001.
- The MathWorks, Inc. MATLAB and SIMULINK, Natick, Mass, USA.
- L. O. Jay, “Solution of index 2 implicit differential-algebraic equations by Lobatto Runge-Kutta methods,” BIT. Numerical Mathematics, vol. 43, no. 1, pp. 93–106, 2003.
- L. C. Jay, “Convergence of Runge-Kutta methods for differential-algebraic systems of index 3,” Applied Numerical Mathematics. An IMACS Journal, vol. 17, no. 2, pp. 97–118, 1995.
- J. D. Lambert, Numerical Methods for Ordinary Differential Systems, John Wiley & Sons, 1991.
- S. Amat and P. Pedregal, “On an alternative approach for the analysis and numerical simulation of sti ODEs,” DCDS A, vol. 33, no. 4, pp. 1275–1291, 2012.
Copyright © 2012 S. Amat et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.