An Autocorrelation Term Method for Curve Fitting

Abstract

The least-squares method is the most popular method for fitting a polynomial curve to data. It is based on minimizing the total squared error between a polynomial model and the data. In this paper we develop a different approach that exploits the autocorrelation function. In particular, we use the nonzero lag autocorrelation terms to produce a system of quadratic equations that can be solved together with a linear equation derived from summing the data. There is a maximum of 2𝑀 solutions when the polynomial is of degree 𝑀. For the linear case, there are generally two solutions. Each solution is consistent with a total error of zero. Either visual examination or measurement of the total squared error is required to determine which solution fits the data. A comparison between the comparable autocorrelation term solution and linear least squares shows negligible difference.

1. Introduction

Curve fitting is the process of fitting a curve, in this case a polynomial, to a set of data points. There are different types of curve fitting, but we will discuss only the most popular method, the method of least squares [1].

It is assumed that the data consists of fluctuations about an ideal curve. These fluctuations create an error between the polynomial model and the actual data. After computing a total squared error, we can apply calculus to minimize the squared error with respect to the coefficients of the polynomial. This produces a set of  linear equations called the normal equations. The coefficients are derived from solving the normal equations [2].

A different approach to the problem is as follows.

We propose that the data we are considering consists of a deterministic component and a random component with zero mean. The random component does not correlate with the deterministic component and it does not correlate with itself at nonzero lag. We want to extract only the portion of the data that is correlative at nonzero lag. This requires the computation of nonzero lag autocorrelation terms and produces a system of quadratic equations [3]. After substitution based on a linear equation derived from summing the data, the system of quadratic equations consists of 𝑀 equations with 𝑀 unknowns, where 𝑀 is the degree of the polynomial. This results in a maximum of 2𝑀 solutions. Each solution is consistent with a total error between the polynomial and the data that is equal to zero. Either visual examination or measurement of the total squared error is required to determine which solution fits the data.

In this paper, we derive the autocorrelation term method and compare it to linear least squares.

2. The Autocorrelation Term Method

Consider a set of data (𝑥_𝑖,𝑦_𝑖)(𝑖=1,2,3,…,𝑁). Characterize the 𝑦 data as the sum of a polynomial function 𝑦(𝑥_𝑖) and a discrete, zero-mean random variable [4] 𝑍_𝑖∶𝑍_𝑖𝑦∶Ω→ℝ,_𝑖=𝑦𝑥_𝑖+𝑍_𝑖. (1) Consequently, we can write 𝑦_𝑖+𝑚𝑦_𝑖=𝑦𝑥_𝑖+𝑚+𝑍_𝑖+𝑚𝑦𝑥_𝑖+𝑍_𝑖,=𝑦𝑥_𝑖+𝑚𝑦𝑥_𝑖+𝑍_𝑖+𝑚𝑦𝑥_𝑖+𝑍_𝑖𝑦𝑥_𝑖+𝑚+𝑍_𝑖+𝑚𝑍_𝑖,𝑚∈{1,2,…}. (2) Let 𝑁≫𝑚.

Summation yields ^𝑁−𝑚_𝑖=1𝑦_𝑖+𝑚𝑦_𝑖=^𝑁−𝑚_𝑖=1𝑦𝑥_𝑖+𝑚𝑦𝑥_𝑖 (3) since the cross-correlation [5] and autocorrelation terms with 𝑍_𝑖 must be zero. The polynomial function of degree 𝑀 can be written as 𝑦𝑥_𝑖=𝑎₀+𝑎₁𝑥_𝑖+𝑎₂𝑥²_𝑖+⋯+𝑎_𝑀𝑥^𝑀_𝑖=^𝑀_𝑗=0𝑎_𝑗𝑥^𝑗_𝑖. (4) Therefore, (3) becomes the system of equations: ^𝑁−𝑚_𝑖=1𝑦_𝑖+𝑚𝑦_𝑖=^𝑁−𝑚_𝑖=1^𝑀_𝑗=0𝑎_𝑗𝑥^𝑗_𝑖+𝑚^𝑀_𝑗=0𝑎_𝑗𝑥^𝑗_𝑖,𝑚=1,2,…,𝑀. (5) Using (1), we can write the average 𝑦_𝑖=𝑦𝑥_𝑖+𝑍_𝑖=𝑦𝑥_𝑖+𝑍_𝑖. (6) We know that ⟨𝑍_𝑖⟩=0, so 𝑦_𝑖=𝑦𝑥_𝑖 (7) which implies 𝑦_𝑖=𝑦𝑥_𝑖 (8) or ^𝑁_𝑖=1𝑦_𝑖=^𝑁𝑀_𝑖=1𝑗=0𝑎_𝑗𝑥^𝑗_𝑖. (9) There are a maximum of 2𝑀 solutions to the system of equations created by (5) and (9) for the coefficients {𝑎₀,𝑎₁,…,𝑎_𝑀}. Observe that the total error, 𝐸, is presumed to be zero: 𝐸=^𝑁_𝑖=1^𝑀_𝑗=0𝑎_𝑗𝑥^𝑗_𝑖−𝑦_𝑖=0. (10)

3. The Least-Squares Straight Line

Before examining the autocorrelation method, it is worth becoming familiar with the special case of fitting a straight line in the least-squares sense. We fit 𝑦(𝑥)=𝑎₀+𝑎₁𝑥 (11) to a set of data (𝑥_𝑖,𝑦_𝑖)(𝑖=1,2,3,…,𝑁). We have two parameters 𝑎₀ and 𝑎₁ and want to minimize the function of two variables (𝑎₀ and 𝑎₁): 𝐹𝑎₀,𝑎₁=^𝑁_𝑖=1𝑦𝑥_𝑖−𝑦_𝑖². (12) This becomes 𝐹𝑎₀,𝑎₁=^𝑁_𝑖=1𝑎₀+𝑎₁𝑥_𝑖−𝑦_𝑖². (13) Apply the calculus method 𝜕𝐹𝑎₀,𝑎₁𝜕𝑎₀=2𝑎₀+𝑎₁𝑥_𝑖−𝑦_𝑖𝑎=0,𝜕𝐹₀,𝑎₁𝜕𝑎₁=2𝑎₀+𝑎₁𝑥_𝑖−𝑦_𝑖𝑥_𝑖=0. (14) Dropping factor 2 and rewriting, we get 𝑎₀𝑁+𝑎₁𝑥_𝑖=𝑦_𝑖,𝑎₀𝑥_𝑖+𝑎₁𝑥²_𝑖=𝑥_𝑖𝑦_𝑖. (15) Solving these equations for 𝑎₀ and 𝑎₁ yields 𝑎₀=1𝑁𝑦_𝑖−1𝑁𝑥_𝑖∑𝑥_𝑖𝑦_𝑖−(1/𝑁)∑𝑦_𝑖∑𝑥_𝑖∑𝑥²_𝑖−∑𝑥_𝑖²,𝑎/𝑁₁=∑𝑥_𝑖𝑦_𝑖−(1/𝑁)∑𝑦_𝑖∑𝑥_𝑖∑𝑥²_𝑖−∑𝑥_𝑖²./𝑁 (16)

4. The Autocorrelation Term Method for 𝑀=1

For 𝑀=1, system (5) becomes ^𝑁−1_𝑖=1𝑦_𝑖+1𝑦_𝑖=^𝑁−1_𝑖=1𝑎₀+𝑎₁𝑥_𝑖+1𝑎₀+𝑎₁𝑥_𝑖=(𝑁−1)𝑎²₀+^𝑁−1_𝑖=1𝑥_𝑖+1+𝑥_𝑖𝑎₀𝑎₁+^𝑁−1_𝑖=1𝑥_𝑖+1𝑥_𝑖𝑎²₁. (17) Suppressing the summation limits, we can write 𝑦_𝑖+1𝑦_𝑖=(𝑁−1)𝑎²₀+𝑥_𝑖+1+𝑥_𝑖𝑎₀𝑎₁+𝑥_𝑖+1𝑥_𝑖𝑎²₁. (18) Equation (9) becomes 𝑦_𝑖=𝑁𝑎₀+𝑎₁𝑥_𝑖. (19) Solving (19) for 𝑎₀ yields 𝑎₀=1𝑁𝑦_𝑖−𝑎₁𝑥_𝑖. (20) Squaring yields 𝑎²₀=1𝑁²𝑦_𝑖²−2𝑎₁𝑦_𝑖𝑥_𝑖+𝑎²₁𝑥_𝑖². (21) Now substitute (20) and (21) into (18) and organize the terms: (𝑁−1)𝑁²𝑥_𝑖²+𝑥_𝑖+1𝑥_𝑖−1𝑁𝑥_𝑖+1+𝑥_𝑖𝑥_𝑖𝑎²₁+1𝑁𝑥_𝑖+1+𝑥_𝑖𝑦_𝑖−2(𝑁−1)𝑁²𝑦_𝑖𝑥_𝑖×𝑎₁+(𝑁−1)𝑁²𝑦_𝑖²−𝑦_𝑖+1𝑦_𝑖=0. (22) This is in the form of a quadratic equation that we can write as 𝐴𝑎²₁+𝐵𝑎₁+𝐶=0, (23) where, with reference to (22), 𝐴 is the precursor of 𝑎²₁, 𝐵 is the precursor of 𝑎₁, and 𝐶 is the residual expression.

The solutions are the roots 𝑎₁=−𝐵±𝐵²−4𝐴𝐶.2𝐴 (24)𝑎₀ is found using (20).

5. The Autocorrelation Term Method for Higher Values of 𝑀

The autocorrelation term method for curve fitting with an 𝑀th degree polynomial requires the solution of one linear equation and 𝑀 quadratic equations with 𝑀+1 variables which reduces to 𝑀 quadratic equations with 𝑀 variables. This can produce 2𝑀 solutions [6]. For example, consider the 𝑀=2 case. System (5) can be written as 𝑦_𝑖+1𝑦_𝑖=𝑎₀+𝑎₁𝑥_𝑖+1+𝑎₂𝑥²_𝑖+1𝑎₀+𝑎₁𝑥_𝑖+𝑎₂𝑥²_𝑖,𝑦_𝑖+2𝑦_𝑖=𝑎₀+𝑎₁𝑥_𝑖+2+𝑎₂𝑥²_𝑖+2𝑎₀+𝑎₁𝑥_𝑖+𝑎₂𝑥²_𝑖. (25) Equation (9) can be written as 𝑦_𝑖=𝑎₀+𝑎₁𝑥_𝑖+𝑎₂𝑥²_𝑖. (26) From (26) we can write 𝑎₀=1𝑁𝑦_𝑖−𝑎₁𝑁𝑥_𝑖−𝑎₂𝑁𝑥²_𝑖. (27) Substitute (27) into (25): 𝑦_𝑖+1𝑦_𝑖1=𝑁𝑦_𝑖+𝑥_𝑖+1−1𝑁𝑥_𝑖𝑎₁+𝑥²_𝑖+1−1𝑁𝑥²_𝑖𝑎₂∗1𝑁𝑦_𝑖+𝑥_𝑖−1𝑁𝑥_𝑖𝑎₁+𝑥²_𝑖−1𝑁𝑥²_𝑖𝑎₂,𝑦_𝑖+2𝑦_𝑖1=𝑁𝑦_𝑖+𝑥_𝑖+2−1𝑁𝑥_𝑖𝑎₁+𝑥²_𝑖+2−1𝑁𝑥²_𝑖𝑎₂∗1𝑁𝑦_𝑖+𝑥_𝑖−1𝑁𝑥_𝑖𝑎₁+𝑥²_𝑖−1𝑁𝑥²_𝑖𝑎₂. (28) Equations (28) are two quadratic equations in 𝑎₁ and 𝑎₂. This can produce four solutions. In general, these types of systems can be solved on a Grobner basis using Buchberger’s algorithm [7].

6. A Comparison of the Autocorrelation Term Method to Linear Least Squares

We have constructed synthetic data consisting of the sum of a line and a random variable. The line has an intercept of fifteen and a slope of five. The random variable is Gaussian with a zero mean. The amplitude of the random variable is twenty. There are a total of one hundred equally spaced data points. Figure 1 shows the data, the least squares fit, and the autocorrelation term fit for 𝑎₁=(−𝐵+𝐵²−4𝐴𝐶)/2𝐴. Figure 2 shows the data, the least squares fit, and the autocorrelation term fit for 𝑎₁=(−𝐵−𝐵²−4𝐴𝐶)/2𝐴. In this example, we find that the quadratic roots are negatives of one another. This occurs because 𝑥_𝑖=𝑖−1 which implies that 𝐵=0. This follows from (22) which for 𝐵=0 implies 𝑥_𝑖+1+𝑥_𝑖=2(𝑁−1)𝑁𝑥_𝑖. (29) This becomes ()=2()2𝑖−1𝑁−1𝑁(𝑖−1) (30) which reduces to ^𝑁−1_𝑖=1(2𝑖−1)=(𝑁−1)². (31)

Figure 1 

An overlay of the data, the least-squares fit, and the autocorrelation term fit for 𝑎1=(−𝐵+𝐵2−4𝐴𝐶)/2𝐴.

Figure 2 

An overlay of the data, the least-squares fit, and the autocorrelation term fit for 𝑎1=(−𝐵−𝐵2−4𝐴𝐶)/2𝐴.

7. Conclusions

We have derived a method for computing polynomial curve fits to data based on terms from the autocorrelation function. The method produces 𝑀 quadratic equations with 𝑀 variables for a polynomial of degree 𝑀. The solutions of this system maximally produce 2𝑀 curves. Each solution is consistent with a total error between the polynomial and the data that is equal to zero. The proper curve can be selected with either visual examination or measurement of the total squared error. We tested this method with a linear curve fit and compared it to linear least squares. There is negligible difference between the comparable solutions.

Acknowledgment

Discussions with Gwen Houston and Dominique Lueckenhoff are greatly appreciated.