A Two-Point Newton Method Suitable for Nonconvergent Cases and with Super-Quadratic Convergence

Abstract

An iterative formula based on Newton’s method alone is presented for the iterative solutions of equations that ensures convergence in cases where the traditional Newton Method may fail to converge to the desired root. In addition, the method has super-quadratic convergence of order 2.414 (i.e., 1+√2). Newton method is said to fail in certain cases leading to oscillation, divergence to increasingly large number, or offshooting away to another root further from the desired domain or offshooting to an invalid domain where the function may not be defined. In addition when the derivative at the iteration point is zero, Newton method stalls. In most of these cases, hybrids of several methods such as Newton, bisection, and secant methods are suggested as substitute methods and Newton method is essentially blended with other methods or altogether abandoned. This paper argues that a solution is still possible in most of these cases by the application of Newton method alone without resorting to other methods and with the same computational effort (two functional evaluations per iteration) like the traditional Newton method. In addition, the proposed modified formula based on Newton method has better convergence characteristics than the traditional Newton method.

1. Introduction

Iterative procedures for solutions of equations are routinely employed in many science and engineering problems. Starting with the classical Newton methods, a number of methods for finding roots of equations have come to exist, each of which has its own advantages and limitations. The Newton method of root finding is based on the iterative formula: 𝑥_𝑘+1=𝑥_𝑘−𝑦𝑥_𝑘𝑦^′𝑥_𝑘. (1) Newton’s method displays a faster quadratic convergence near the root while it requires evaluation of the function and its derivative at each step of the iteration. However, when the derivative evaluated is zero, Newton method stalls. For low values of the derivative, the Newton iteration offshoots away from the current point of iteration and may possibly converge to a root far away from the intended domain. For certain forms of equations, Newton method diverges or oscillates and fails to converge to the desired root. In addition, the convergence of Newton method can be slow near roots of multiplicity although modifications can be made to increase the rate of convergence [1].

Modifications of the Newton method with higher order convergence have been proposed that require also evaluation of a function and its derivatives. An example of such methods is a third order convergence method by Weerakoon and Fernando [2] that requires evaluation of one function and two first derivatives. A fourth order iterative method, according to Traub [3] also requires evaluation of one function and two derivative evaluations. Grau-Sánchez and Díaz-Barrero [4] gave a compositing of function evaluation at a point and its derivative to improve the convergence of Newton’s method from 2 to 4. Recently other methods of fifth, sixth, seventh, and higher order convergence have been proposed [5–11]. While higher order Newton methods ensure faster convergence, their stability for certain equation forms may have the same problem similar to that of the traditional Newton method.

The secant method does not require evaluation of derivatives. However, the rate of convergence is about 1.618, the convergence may be a problem for some forms of equations, and the secant method may fail to converge in those cases. Muller’s method is an extension of the secant method to a quadratic polynomial [12]. It requires three functional evaluations to start with but continues with one function evaluation afterwards. The method does not require derivatives and the rate of convergence near the root is superlinear, that is, about 1.84. However, Muller’s method can converge to a complex root from an initial real number [13]. Muller’s method also requires the three points to be distinct. If two of the points coincide, the method degenerates to secant method.

Hybrids of methods are also used to provide stability and ensure convergence to a desired root. For example, Newton or secant methods can be combined with bisection to bracket the root by a small interval so that a good initial guess is available for applying Newton method with quadratic convergence. Dekker’s method [14] combines bisection method with that of secant method. The method starts by bracketing the root between two initial points that have functional values opposite in sign. The secant estimate of 𝑥 is compared with the bisection of the interval, the one estimate resulting in small interval with the point of smaller magnitude functional value is chosen, and the iteration continues. Brent’s method [15] is a root finding algorithm that combines root bracketing, bisection, and inverse quadratic interpolation. It is a modification of Dekker’s method to avoid slow convergence when the difference between consecutive estimates of 𝑥 is arbitrarily small. In such cases bisection is used for the next root estimate.

The Leap-frogging Newton method [16] uses the Newton method as an intermediate step for the estimation of the root followed by the application of the secant method using the initial point and the intermediate point found by Newton method. The method has cubic convergence and works in certain pathological cases where Newton method may fail. However, being dependent on Newton method for the intermediate step, the method may suffer from the same drawbacks of using the traditional Newton method mentioned above.

2. Method Development

It will be shown that the iterative formula for the two-point method will take the form 𝑥_𝑘+1=𝑥_𝑘−1−𝑥_𝑘−1−𝑥_𝑘1−𝑦_𝑘/𝑦_𝑘−1𝑦_𝑘−𝑦_𝑘−1/𝑥_𝑘−𝑥_𝑘−1/𝑦^′_𝑘. (2) The method starts by selecting two points lying on a curve 𝑦=𝑓(𝑥), namely  (𝑥₀,𝑦₀)  and  (𝑥₁,𝑦₁). A line connecting the two points is drawn from the point  (𝑥₀,𝑦₀)  to the point  (𝑥₁,𝑦₁)  as shown in Figure 1. A new variable  𝑚  (the cotangent of the angle 𝛼₁ between this line and the vertical) is defined so that 𝑚₁=cot𝛼₁=𝑦₀−𝑦₁𝑥₀−𝑥₁. (3) In general, for any other point (𝑥,𝑦) lying on the curve, the variable  𝑚  is defined so that: (𝛼)=𝑦𝑚=cot⁰−𝑦𝑥₀.−𝑥 (4) Next, Newton method of finding roots will be applied taking  𝑚  as the independent variable and 𝑦 as the dependent variable. Denoting that 𝑚_𝑟 is the estimate of the root for which  𝑦(𝑚)=0  from the Newton method with the corresponding value of  𝑥_𝑟 and applying Newton method gives 𝑚_𝑟=𝑚₁−𝑦₁𝑑𝑦/𝑑𝑚₁. (5) Substituting for 𝑚_𝑟 and 𝑚₁ the equivalent expressions in terms of 𝑥 and 𝑦 values, namely, 𝑚_𝑟=𝑦₀−0𝑥₀−𝑥_𝑟,𝑚₁=𝑦₀−𝑦₁𝑥₀−𝑥₁ (6) into (5) above gives the following expression: 𝑦₀𝑥₀−𝑥_𝑟=𝑦₀−𝑦₁𝑥₀−𝑥₁−𝑦₁𝑑𝑦/𝑑𝑚₁. (7) The derivative 𝑑𝑦/𝑑𝑚 is evaluated from 𝑑𝑦/𝑑𝑥=𝑦^′ and 𝑑𝑚/𝑑𝑥 using the formula 𝑑𝑦=𝑑𝑚𝑑𝑦/𝑑𝑥=𝑦𝑑𝑚/𝑑𝑥^′.𝑑𝑚/𝑑𝑥 (8) The evaluation of the derivative 𝑑𝑚/𝑑𝑥 in turn gives 𝑑𝑚=𝑑𝑑𝑥𝑦𝑑𝑥⁰−𝑦𝑥₀=𝑦−𝑥⁰−𝑦−𝑥₀−𝑥𝑦^′𝑥₀−𝑥². (9) Therefore, 𝑑𝑦/𝑑𝑚 can now be written in terms of 𝑥,  𝑦 and 𝑦^′ as follows: 𝑑𝑦=𝑦𝑑𝑚^′𝑥₀−𝑥²𝑦₀−𝑦−𝑥₀−𝑥𝑦^′. (10) Substituting the above expression for 𝑑𝑦/𝑑𝑚 in the equation 𝑦₀𝑥₀−𝑥_𝑟=𝑦₀−𝑦₁𝑥₀−𝑥₁−𝑦₁𝑑𝑦/𝑑𝑚₁ (11) gives 𝑦₀𝑥₀−𝑥_𝑟=𝑦₀−𝑦₁𝑥₀−𝑥₁−𝑦₁𝑦^′𝑥₀−𝑥²/𝑦₀−𝑦−𝑥₀−𝑥𝑦^′. (12) Solving for the root estimate 𝑥_𝑟 and further rearranging results in 𝑥_𝑟=𝑥₀−𝑥₀−𝑥₁1−𝑦₀/𝑦₁𝑦₁−𝑦₀/𝑥₁−𝑥₀/𝑦^′₁. (13) Continuing the iteration using the above formula, for the 𝑘th step of the iteration, the  𝑘+1th estimate of the root will take the form 𝑥_𝑘+1=𝑥_𝑘−1−𝑥_𝑘−1−𝑥_𝑘1−𝑦_𝑘/𝑦_𝑘−1𝑦_𝑘−𝑦_𝑘−1/𝑥_𝑘−𝑥_𝑘−1/𝑦^′_𝑘. (14) Denoting (𝑦_𝑘−𝑦_𝑘−1)/(𝑥_𝑘−𝑥_𝑘−1)  by  (Δ𝑦/Δ𝑥)_𝑘 will result in the expression 𝑥_𝑘+1=𝑥_𝑘−1−𝑥_𝑘−1−𝑥_𝑘1−𝑦_𝑘/𝑦_𝑘−1Δ𝑦/Δ𝑥_𝑘/𝑦^′_𝑘. (15)

Figure 1 

The graph of 𝑦=𝑓(𝑥) on which the angle  𝛼  is defined between the two points.

3. Proof of Super-Quadratic Convergence

Recalling the iteration formula of (14), 𝑥_𝑘+1=𝑥_𝑘−1−𝑥_𝑘−1−𝑥_𝑘1−𝑦_𝑘/𝑦_𝑘−1𝑦_𝑘−𝑦_𝑘−1/𝑥_𝑘−𝑥_𝑘−1/𝑦^′_𝑘. (16) Defining the error at the 𝑘th iteration to be 𝑒_𝑘=𝑥_𝑘−𝑟, where  𝑟  is the root of the equation desired, the errors at the 𝑘−1th and 𝑘th iteration are also defined similarly as follows: 𝑒_𝑘−1=𝑥_𝑘−1−𝑟,𝑒𝑘₊₁=𝑥_𝑘+1−𝑟. (17) The iteration formula in (14) can now be rewritten in terms of the error terms as follows: 𝑒_𝑘+1=𝑒_𝑘−1−𝑒_𝑘−1−𝑒_𝑘1−𝑦_𝑘/𝑦_𝑘−1𝑦_𝑘−𝑦_𝑘−1/𝑒_𝑘−𝑒_𝑘−1/𝑦^′_𝑘. (18) Expanding the 𝑦_𝑘−1, 𝑦_𝑘 and 𝑦^′_𝑘, terms about the root 𝑟 using Taylor series expansion, 𝑦_𝑘=𝑦_𝑟=0+𝑒_𝑘𝑦^′_𝑟+𝑒²_𝑘𝑦_𝑟^′′2+𝑒³_𝑘𝑦_𝑟^′′′6+𝑂𝑒_𝑘⁴,𝑦_𝑘=𝐶₁𝑒_𝑘+𝐶₂𝑒²_𝑘+𝐶₃𝑒³_𝑘+𝑂𝑒_𝑘⁴, (19) where 𝐶_𝑛=𝑦^𝑛_𝑟/𝑛!  for  𝑛=1,2,3,….

The Taylor series expansions of  𝑦_𝑘−1  and 𝑦^′_𝑘 about 𝑥=𝑟 are formulated similarly as follows: 𝑦_𝑘−1=𝐶₁𝑒_𝑘−1+𝐶₂𝑒²_𝑘−1+𝐶₃𝑒³_𝑘−1+𝑂𝑒_𝑘−1⁴,𝑦^′_𝑘=𝐶₁+𝐶₂𝑒_𝑘+𝐶₃𝑒²_𝑘+𝐶₄𝑒³_𝑘+𝑂𝑒_𝑘⁴. (20) The algebraic expression of (18), after substituting the previous Taylor series forms, was simplified using MATLAB program. After, the fourth order error terms in 𝑒_𝑘 and 𝑒_𝑘−1 were discarded and the error terms of the denominator were also discarded compared to the dominant term of 𝐶²₁, the resulting expression will be as follows: 𝑒_𝑘+1=𝑒_𝑘−1−𝑒_𝑘−1−𝑒_𝑘1−𝑦_𝑘/𝑦_𝑘−1𝑦_𝑘−𝑦_𝑘−1/𝑒_𝑘−𝑒_𝑘−1/𝑦^′_𝑘=𝐶²₂−𝐶₁𝐶₃𝐶²₁𝑒²_𝑘𝑒_𝑘−1. (21) Therefore, the error sequence has the simplified expression 𝑒_𝑘+1=𝐶²₂−𝐶₁𝐶₃𝐶²₁𝑒²_𝑘𝑒_𝑘−1. (22) Defining positive real terms 𝑆_𝑘 and 𝑆_𝑘−1 so that 𝑆_𝑘=𝑒_𝑘+1𝑒^𝛼_𝑘,𝑆_𝑘−1=𝑒_𝑘𝑒^𝛼_𝑘−1,𝑒_𝑘+1=𝑆_𝑘𝑆^𝛼_𝑘−1𝑒^𝛼2_𝑘−1,𝑒_𝑘+1𝑒²_𝑘𝑒_𝑘−1=𝐶²₂−𝐶₁𝐶₃𝐶²₁=𝑆_𝑘𝑆^𝛼−2_𝑘−1𝑒^{𝛼2−2𝛼−1}_𝑘−1. (23) Near the root, the above expression approaches the constant terms in  𝐶. Therefore, the power of the error term 𝛼²−2𝛼−1 shall approach zero: 𝛼²√−2𝛼−1=0,𝛼=2±4+42=1±√2≈2.414. (24) The positive solution of  𝛼  is the one for which the error sequence converges to zero. Therefore, the iteration formula of (14) near the root has a convergence of order 2.414 which is super-quadratic.

4. Stability of the Proposed Method

The proposed method shows stability near points where application of the traditional Newton method may result in oscillation, divergence, or offshooting away from the desired root or offshooting to a possibly invalid domain where the function 𝑦=𝑓(𝑥) may not be defined. This stability characteristic may be studied by examining the iteration formula again which takes the following form: 𝑥_𝑘+1=11−𝑟𝑥_𝑘−1+1𝑟𝑥_𝑘, (25) where the variable  𝑟  is defined as 𝑦𝑟=1−^𝑘𝑦_𝑘−1𝑦_𝑘−𝑦_𝑘−1/𝑥_𝑘−𝑥_𝑘−1𝑦^′_𝑘. (26) The estimate for the root at the 𝑘+1th iteration, 𝑥_𝑘+1, therefore, can be taken as the weighted sum of the 𝑥_𝑘−1 and 𝑥_𝑘 values at the 𝑘−1th and 𝑘th iteration, respectively. The weighing factors are (1−1/𝑟) for 𝑥_𝑘−1 and (1/𝑟) for 𝑥_𝑘. When the method converges to the root, the value of 𝑟 approaches unity. This is found by examining the behavior of 𝑟 near the root and noting that 𝑦_𝑘 approaches 0 near the root; that is, lim_𝑘→∞(𝑟)0=1−𝑦_𝑘−1𝑦_𝑘−𝑦_𝑘−1/𝑥_𝑘−𝑥_𝑘−1𝑦^′_𝑘=1−0=1. (27) The iterative formula will, near the root, approach the following expression: lim_𝑘→∞𝑥_𝑘+1=11−1𝑥_𝑘−1+11𝑥_𝑘=(0)𝑥_𝑘−1+(1)𝑥_𝑘. (28) The iteration, therefore, moves away from 𝑥_𝑘−1 with a weighing factor approaching zero and gives weighing factor of 1 to 𝑥_𝑘.

Near points where the derivative of the function may approach zero, the  𝑟  value approaches infinity and the root estimate weighs heavily in favor of 𝑥_𝑘−1 rather than 𝑥_𝑘. This can be seen from the following limit evaluation of 𝑟: lim_{𝑑𝑦/𝑑𝑥→0}(𝑟)𝑦=1−^𝑘𝑦_𝑘−1𝑦_𝑘−𝑦_𝑘−1/𝑥_𝑘−𝑥_𝑘−10=1±∞=±∞,lim_{𝑑𝑦/𝑑𝑥→0}𝑥_𝑘+1=11−∞𝑥_𝑘−1+1∞𝑥_𝑘=(1)𝑥_𝑘−1+(0)𝑥_𝑘. (29) The iteration, therefore, moves away from 𝑥_𝑘 with a weighing factor approaching zero and gives weighing factor of near to 1 to 𝑥_𝑘−1.

Near points where the derivative of the function may approach zero, Newton method typically displays oscillation or offshooting behavior. The proposed two-point modified Newton procedure however shows stability without displaying oscillation and offshooting tendencies. For example, for the function 𝑦=sin(𝑥), the derivative of  𝑦  (i.e., 𝑑𝑦/𝑑𝑥=cos(𝑥)) is zero at 𝑥=𝜋/2. Starting Newton method near this point, say 𝑥=(𝜋/2+0.01), will result in offshooting of the iteration to a different domain.

As shown in Figure 2, for a starting point  𝑥=1.58079633  where the function 𝑦=sin(𝑥) has near zero derivative, the traditional Newton method of shoots to a root further from the nearest root of 𝑥=0. The Newton method as such converges to 𝑥=100.531. On the other hand, the proposed two-point Newton method shows stability and converges to the nearest root of 𝑥=0.

(a)

(b)

Figure 2 

Stability of the proposed two-point Newton method near points of zero derivatives.

Another example of the stability of the proposed method is when Newton method diverges instead of converging to the desired root. Figures 3 and 4 show comparison of the iterative values of  𝑥  for the proposed method and the traditional Newton method for the function 𝑦=𝑥^1/3 which is often used to illustrate the pathological condition with respect to application of Newton’s method which leads to an oscillating divergence to infinity. By contrast the proposed method displays an oscillating convergence to the desired root albeit with slower rate of convergence of order 1.0.

Figure 3 

Application of Newton method to 𝑦=𝑥1/3 leading to oscillating divergence.

Figure 4 

Application of the proposed two-point Newton method to 𝑦=𝑥1/3 leading to oscillating convergence.

5. Application Examples

Equations used to test efficiency of root finding methods are used here to evaluate the number of iterations required to reach to a specified level of convergence (Table 1). The stopping criterion used for the iteration process is given by 𝑥_𝑘−𝑥_𝑘−1+𝑦_𝑘<10⁻¹⁵. (30) The rate of convergence towards the root 𝑥=𝑟 for each step of the iteration is evaluated using the formula 𝛼_𝑘=Log𝑒_𝑘+1Log𝑒_𝑘=Log𝑥_𝑘+1−𝑟Log𝑥_𝑘−𝑟. (31) Table 1 shows comparison of the proposed two-point Newton method with the Newton and secant methods for a number of equations used to test efficiency of root finding methods elsewhere. A super-quadratic convergence with which the proposed method converges to the root is mostly evident with  𝛼_𝑘  values being close to 2.414 during most of the iterations. It can also be seen from Table 1 that a less number of iterations are required to reach convergence for the proposed method than those required for Newton and secant methods. For example, for equations with multiple roots such as 𝑦=(𝑥−1)⁶−1, shown in Table 1, Newton method displays linear convergence while the proposed method converges super-quadratically with an order of 2.414. Similar order of convergence is also observed for the equation 𝑦=𝑒^{𝑥2+7𝑥−30}−1.

Table 1 

Comparison of results of iterations of the two-point Newton method with Newton and secant methods.

5.1. Examples Where the Proposed Method Works While Newton Method Fails

The advantage of the use of the proposed two-point Newton method is shown for cases where the Newton method and in several cases also the secant method fail to converge to the root. Table 2 shows the results of the iteration for several examples of equations. In all of the examples listed, Newton method fails to converge whereas the proposed two-point Newton method converges. Secant method also fails to converge in several of the examples cited in Table 2. For example, in the case of  𝑦=Log(𝑥), Newton method starting with 𝑥=3 fails with the second iteration because the estimated 𝑥 value is a negative number whose logarithm is undefined. For the case  𝑦=arctan(𝑥)  Newton method always diverges to increasingly large number for any starting  𝑥  value while the proposed two-point Newton method converges super-quadratically near the root. For  𝑦=𝑥^1/3, Newton method also diverges whereby each iteration gives an estimate of the root which is twice the previous value and with alternate signs leading to oscillating divergence. On the other hand application of the proposed two-point modification of Newton method leads to oscillating convergence at somehow reduced rate of convergence (of order 1 as explained above).

Table 2 

Results of application of the proposed method for cases in which Newton or secant method fails to converge.

For the equation  𝑦=10𝑥𝑒^−𝑥2−1, both of Newton and secant methods continue to diverge to increasingly large 𝑥 values while the proposed two-point Newton method shows stability of convergence. For the polynomials of different degrees cited in the Table 2, Newton and secant methods display oscillation for the starting values shown in the Table 2 while the proposed two-point Newton method displays stability and super-quadratic convergence for iteration near the root.

6. Conclusion

A numerical procedure of root finding using two-point modification of Newton method has been presented. It is proved that the method has a super-quadratic convergence of order about 2.414. The method is based on application of Newton iteration formula by taking as the independent variable the cotangent of the angle between the line connecting the two successive points of iteration with the vertical and as the dependent variable the given function 𝑦=𝑓(𝑥). The resulting iteration formula for root estimation is shown to be the weighted sum of the estimates of the two previous iterations with a weighing factor that penalizes the iteration point having undesirable characteristics such as a near zero derivative. For example, near a point where the derivative is zero, the weighing factor for that point will be near zero effectively moving the iteration away from that undesirable point.

Application examples have been given to demonstrate that the proposed method requires fewer number of iterations for convergence to a root than the traditional Newton and secant methods. The method offers a particular advantage for cases where the traditional Newton method and its variants of various order convergence may not converge. A number of examples are given where the proposed method converges to a root in a stable manner without oscillation, divergence, or offshooting from the desired domain of the root, whereas the Newton method and in several of the cases also the secant method display undesirable behaviors which prevent convergence to the desired root.

In terms of computational effort, the proposed method requires one function evaluation and one derivative evaluation at each step of the iteration except for the first step where two functional evaluations are required. The proposed method, therefore, requires essentially the same number of functional evaluations as the traditional Newton method while offering a super-quadratic convergence of order 2.414.