Affine Projection Algorithm Using Regressive Estimated Error

Abstract

An affine projection algorithm using regressive estimated error (APA-REE) is presented in this paper. By redefining the iterated error of the affine projection algorithm (APA), a new algorithm is obtained, and it improves the adaptive filtering convergence rate. We analyze the iterated error signal and the stability for the APA-REE algorithm. The steady-state weights of the APA-REE algorithm are proved to be unbiased and consist. The simulation results show that the proposed algorithm has a fast convergence rate compared with the APA algorithm.

1. Introduction

Adaptive filtering techniques are widely used in noise and echo cancellation, equalization, and beamforming. The normalized least mean square (NLMS) algorithm is a widely used adaptation algorithm due to its computational simplicity and ease of implementation [1]. Unfortunately, for highly colored input signals—with a covariance matrix that exhibits a large dynamic range of eigenvalues—this algorithm deteriorates its convergence rate. To overcome this problem, Ozeki and Umeda [2] discovered the affine projection algorithm (APA) using affine subspace projections. Rupp [3], based on the direction vector, gave a definition for the APA algorithm, and it shows the desirable decorrelation properties. Sankaran and Beex [4], based on the idea that the best improvement in weights occurred while the successive input vectors were orthogonal to each other, proposed the NLMS with orthogonal correction factors (NLMS-OCF). The entire class of algorithms can be referred to as the APA algorithms. In this paper, we will use the formulation by [3] to derive a new adaptive filtering algorithm using regressive estimated error and analyze its convergence rate.

The iterated error of the APA algorithm, which is the same with the estimated output error, is not only caused by the iterated direction, but also caused by the other directions that do not contribute to the iterated direction. In this paper, the regressive estimated error is used as the iterated error to solve this problem, and then we obtain the affine projection algorithm using regressive estimated error (APA-REE). Under the measurement noise-free condition, the iterated error, which is different from the estimated output error, is almost only caused by the iterated direction. The stability for the APA-REE algorithm is also analyzed, and the steady-state weights are proved to be unbiased and consist. The simulation results show that the proposed algorithm has a fast convergence rate compared with the APA algorithm.

2. APA-REE Algorithm

Figure 1 shows an adaptive filter used in the system identification mode, the wide-sense stationary input process {𝑥_𝑛}, which is zero mean, and the corresponding measured output {𝑑_𝑛}, possibly contaminated with the measurement noise {𝜀_𝑛} that is zero-mean white noise, is measurable. The input process is converted into input vectors {𝐱_𝑛}, via a tapped delay line (TDL), and is defined as𝐱_𝑛=𝑥_𝑛𝑥_𝑛−1⋯𝑥_{𝑛−𝑁+1}^𝑇.(1) The objective is to estimate an 𝑁-dimensional weight vector {𝐰_𝑛} using the most recent (𝑚+1) input vectors available at the 𝑛th instant; the latter define the input matrix {𝐗_𝑛,𝑚+1} as follows𝐗_𝑛,𝑚+1=𝐱_𝑛𝐱_𝑛−1⋯𝐱_𝑛−𝑚.(2) The estimated output {𝑑_𝑛} is obtained according to𝑑_𝑛=𝐰^𝐻𝑛𝐱_𝑛.(3) The weight vector {𝐰_𝑛} is adjusted so that the estimated output {𝑑_𝑛} is as close as possible to the measured output {𝑑_𝑛} in the mean square error (MSE) sense. The APA-REE algorithm update of the weight vector {𝐰_𝑛} is given as follows:𝑒𝑒_𝑛̃=𝑑_𝑛−𝑑_𝑛,(4a)_𝑛=𝑒_𝑛̃−^𝑚_𝑖=1𝐰𝑎̂_𝑛,𝑖𝑒̃_𝑛−𝑖,(4b)_𝑛+1=𝐰_𝑛+𝜇𝑒^∗𝑛𝝋^𝐻𝑛𝝋_𝑛𝝋_𝑛,(4c) where the notations (⋅)^∗ and (⋅)^𝐻 are adopted to denote the complex conjugate for the scalar and the Hermitian transpose of a vector or matrix, respectively. 𝑒_𝑛̃ denotes the estimated output error, and 𝑒_𝑛, used as the iterated error of the APA-REE algorithm, is the regressive estimated error. The direction vector {𝝋_𝑛} is evaluated according to 𝝋_𝑛=𝐱_𝑛−𝐗_{𝑛−1,𝑚}𝜶_𝑛=𝐱_𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝐱_𝑛−𝑖.(4d) The vector 𝜶_𝑛=[𝑎̂_𝑛,1𝑎̂_𝑛,2⋯𝑎̂_𝑛,𝑚]^𝑇, which is also used in (4b), is found according to the following least squares formulation:𝜶_𝑛=𝐗^{𝐻𝑛−1,𝑚}𝐗_{𝑛−1,𝑚}⁻¹𝐗^{𝐻𝑛−1,𝑚}𝐱_𝑛.(4e)

Figure 1 

Adaptive system identification scenario.

When the iterated error of formulation (4c) is directly driven by the estimated output error, this algorithm is called APA algorithm, which is firstly introduced by [2]. But the iterated error of the APA-REE algorithm is driven by the regressive estimated error, and the latter will certify that the iterated error is almost only caused by the iterated direction. From (4c), we can also find that the iterated direction of the APA-REE algorithm is the direction vector {𝝋_𝑛}.

3. Regressive Estimated Error Analysis

In order to make the analysis becoming tractable, a true adaptive filter weight vector 𝐰⁰ of dimension 𝑁 is supposed to exist such that the estimated weight error𝐰_𝑛=𝐰⁰−𝐰_𝑛(5) and the corresponding measured output signal𝑑_𝑛=𝐰^0𝐻𝐱_𝑛+𝜀_𝑛,(6) where {𝜀_𝑛} denotes the measurement noise, which is zero mean white noise. Using (4a) into (4b), we obtain𝑒_𝑛=𝑑_𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝑑_𝑛−𝑖−𝑑^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝑑_𝑛−𝑖,(7) where 𝑎̂_𝑛,𝑖, 1≤𝑖≤𝑚 is an element of the vector 𝜶_𝑛. Using (6) and (3) into (7) yields𝑒_𝑛=𝐰^0𝐻𝐱_𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝐰^0𝐻𝐱_𝑛−𝑖−𝐰_𝐻𝑛𝐱_𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝐰^{𝐻𝑛−𝑖}𝐱_𝑛−𝑖+𝜀^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖.(8) The estimated weight vector changes inapparently during the most recent past 𝑚 steps, especially for steady-state. So we can consider that the estimated weight vectors 𝐰_𝑛≈𝐰_𝑛−𝑖, 1≤𝑖≤𝑚, and then, from (8), we obtain𝑒_𝑛=𝐰^0𝐻𝐱^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝐱_𝑛−𝑖−𝐰^𝐻𝑛𝐱^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝐱_𝑛−𝑖+𝜀_𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖.(9) Combining (4d) and (9) yields𝑒_𝑛=𝐰^0𝐻𝝋_𝑛−𝐰^𝐻𝑛𝝋_𝑛+𝜀_𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖.(10) Based on (5) and (10), it follows that𝑒_𝑛=𝐰^𝐻𝑛𝝋_𝑛+𝜀_𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖.(11) From (11), we observe that, if the measurement noise is assumed to be absent, the iterated error of the APA-REE algorithm is only caused by the direction vector, which is the same as the iterated direction.

Now we consider the APA algorithm. Under the assumption that there exists a true adaptive filter weight vector 𝐰⁰ of dimension 𝑁, based on (5) and [5], the iterated error of the APA algorithm can be written as𝑒_𝑛̂=𝐰^𝐻𝑛𝐱_𝑛+𝜀_𝑛.(12) Under the measurement noise-free condition, from (4c) and [3], we can find that both of the iterated directions of the APA-REE and APA algorithms are the direction vector {𝝋_𝑛}. Based on (11), it can be concluded that the iterated error {𝑒_𝑛} of the APA-REE algorithm is only caused by the direction {𝝋_𝑛}, which is the same as the iterated direction {𝝋_𝑛}. But from (12), we can find that the iterated error {𝑒_𝑛̂} of the APA algorithm is caused by the direction {𝐱_𝑛}, which is different from the iterated direction {𝝋_𝑛}. It shows that the iterated error {𝑒_𝑛̂} of the APA algorithm is not only caused by the iterated direction, but also caused by the other directions that do not contribute the iterated direction. So the APA-REE algorithm should have a faster convergence rate compared with the APA algorithm, which will be proved by the latter simulation results.

4. Stability Analysis

To analyze the convergence behavior of (4c), first, substitute (11) into (4c)𝐰_𝑛+1=𝐰_𝑛+𝜇𝝋^𝑛𝝋^𝐻𝑛𝝋^𝐻𝑛𝝋_𝑛𝐰_𝑛+𝜇𝝋^𝑛𝝋^𝐻𝑛𝝋_𝑛𝜀^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖^∗.(13) Combining (13) and (5), the adaptation equation in error form is obtained as𝐰_𝑛+1=𝐰_𝑛−𝜇𝝋^𝑛𝝋^𝐻𝑛𝝋^𝐻𝑛𝝋_𝑛𝐰_𝑛−𝜇𝝋^𝑛𝝋^𝐻𝑛𝝋_𝑛𝜀^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖^∗.(14) Under the assumption that the measurement noise {𝜀_𝑛} is zero-mean white noise, it is independent of the input process {𝑥_𝑛} and therefore of the direction vector {𝝋_𝑛} as well. So the expected value of the last term of (14) becomes𝐸𝜇𝝋^𝑛𝝋^𝐻𝑛𝝋_𝑛𝜀^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖^∗=0.(15) From (14), it can be found that the weight error {𝐰_𝑛} does not depend on the direction vector {𝝋_𝑛}, but it does depend on the direction vector {𝝋_𝑛−1}. So based on (15), taking expectation on both sides of (14) yields𝐸𝐰_𝑛+1=𝐈−𝜇𝐸𝝋^𝑛𝝋^𝐻𝑛𝝋^𝐻𝑛𝝋_𝑛𝐸𝐰^𝑛,(16) where 𝐈 is an identity matrix. Based on [5], under the situation that the input process {𝑥_𝑛} is assumed to be zero-mean wide-sense stationary autoregressive inputs of order 𝑝 and the number of the most recent input vectors 𝑚≥𝑝, the following result can be obtained: 𝐸𝝋^𝑛𝝋^𝐻𝑛𝝋^𝐻𝑛𝝋_𝐧=𝑁−1𝑚−2𝐈.(17) From (16) and (17), we obtain𝐸𝐰_𝑛+1=1−𝜇1𝑁−𝑚−2𝐸𝐰^𝑛.(18) The mean of the weight error will converge to zero if|||1−𝜇1𝑁−𝑚−2|||<1(19) which leads to−1<1−𝜇1𝑁−𝑚−2<1.(20) The right inequality in (20) is unrestrictive; the left inequality produces𝜇<2(𝑁−𝑚−2).(21)

5. Steady-State Weights

Substituting (7) into (4c), we obtain𝐰_𝑛+1=𝐰_𝑛+𝜇𝝋^𝑛𝝋^𝐻𝑛𝝋_𝑛×𝑑_𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝑑_𝑛−𝑖−𝑑^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝑑_𝑛−𝑖^∗.(22) Letting 𝑛→∞, the result 𝐰_𝑛+1=𝐰_𝑛=𝐰_𝑛−𝑖=𝐰_∞, 1≤𝑖≤𝑚 can be obtained. Based on it, (22) can be rewritten as𝝋_𝑛𝑑^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝑑_𝑛−𝑖^∗=𝝋_𝑛𝑑^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝑑_𝑛−𝑖^∗.(23) Using (3) into (23) yields𝝋_𝑛𝐱^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝐱_𝑛−𝑖^𝐻𝐰_∞=𝝋_𝑛𝑑^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝑑_𝑛−𝑖^∗.(24) Combining (4d) and (24), we obtain𝝋_𝑛𝝋^𝐻𝑛𝐰_∞=𝝋_𝑛𝑑^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝑑_𝑛−𝑖^∗.(25) When 𝐰_∞ is independent of the direction vector {𝝋_𝑛}, taking the expectation on both sides of (25) yields 𝐰_∞=𝐸𝝋_𝑛𝝋^𝐻𝑛⁻¹𝐸𝝋_𝑛𝑑^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝑑_𝑛−𝑖^∗.(26) Under the assumption that there exist a true adaptive filter weight vector 𝐰⁰ of dimension 𝑁, and from (6), we obtain𝐰_∞=𝐸𝝋_𝑛𝝋^𝐻𝑛⁻¹×𝐸⎡⎢⎢⎣𝝋_𝑛𝐱^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝐱_𝑛−𝑖^𝐻𝐰⁰+𝝋_𝑛𝜀^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖^∗⎤⎥⎥⎦.(27) From (4d) and (27), the following result can be obtained:𝐰_∞=𝐸𝝋_𝑛𝝋^𝐻𝑛⁻¹×𝐸𝝋_𝑛𝝋^𝐻𝑛𝐰⁰+𝐸𝝋_𝑛𝜀^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖^∗.(28) From (28), it yields𝐰_∞=𝐰⁰+𝐸𝝋_𝑛𝝋^𝐻𝑛⁻¹𝐸𝝋_𝑛𝜀^𝑛−^𝑚_𝑖=1𝑎̂_𝑛,𝑖𝜀_𝑛−𝑖^∗.(29) When the measurement noise is assumed to be zero-mean white noise, the last term of (29) is equal to zero. So we can obtain the result 𝐰_∞=𝐰⁰. And it can be concluded that the steady-state weights of the APA-REE algorithm are unbiased and consist. Furthermore, based on [5], if the input process {𝑥_𝑛} is assumed to be zero-mean wide-sense stationary autoregressive inputs of order 𝑝 and the number of the most recent input vectors 𝑚≥𝑝, the direction vector {𝝋_𝑛} and the estimated weight vector {𝐰_𝑛} are statistically independent, and the direction vector {𝝋_𝑛} is a zero-mean Gaussian random vector, so the results of (15) and (29) can also be obtained to be equal to zeros.

6. Comparison with Simulation Results

In this section, we compare the MSE learning curves between the APA-REE and APA algorithms. The estimated output error of the APA-REE and APA algorithms is defined as in (4a). Simulation results corresponding to two different types of signals, namely, reasonably colored and highly colored, are shown. The reasonably and highly colored signals are generated as a Gaussian first-order autoregressive process with poles at 0.25 and 0.95, respectively. The initial estimate for the weight vector is 𝐰₀=0. The system to be identified has a 32-point long impulse. The simulation results shown are obtained by ensemble averaging over 100 independent trials for each experiment. And we use the maximum entropy assumption for the optimal true weight vector 𝐰⁰. That is, 𝐰⁰ has equal components along all eigenvectors of 𝐸[𝝋_𝑛𝝋^𝐻𝑛]=𝐕Λ𝐕^𝐻:𝐰⁰=𝐕𝟏,(30) where the vector 𝟏=[11⋯1]^𝑇. And the measurement noise {𝜀_𝑛} is assumed to be zero-mean white noise.

Using a reasonably colored input signal, Figure 2 gives the comparison result between the APA-REE and APA algorithms. The measurement noise is assumed to be absent. We use the parameter 𝑚=1. The parameter step size 𝜇 is set to be equal to 0.05. We observe that the APA-REE algorithm improves the adaptive filtering convergence rate than the APA algorithm.

Figure 2 

Reasonably colored input without measurement noise.

Figure 3 considers the same input and parameters as Figure 2, but with the signal-to-noise ratio 60 dB. We observe that the convergence rate of the APA-REE algorithm is slightly better compared with the APA algorithm.

Figure 3 

Reasonably colored input with measurement noise.

Under the situation that the input is a highly colored input signal without the measurement noise, Figure 4 gives the comparative simulation results between the APA-REE and APA algorithms. We use the parameter 𝑚=3 and step size 𝜇=0.2. We find that, for the convergence rate, the APA-REE algorithm is much better than the APA algorithm.

Figure 4 

Highly colored input without measurement noise.

Figure 5 considers the same input and parameters as Figure 4, but the signal-to-noise ratio is set to be equal to 60 dB. From Figure 5, we observe that the APA-REE algorithm converges faster than the APA algorithm.

Figure 5 

Highly colored input with measurement noise.

7. Conclusions

The iterated error of the APA algorithm is not only caused by the iterated direction {𝝋_𝑛}, but also caused by the other directions that do not contribute to the iterated direction. In this paper, the APA-REE algorithm is presented to ameliorate this problem. Under the measurement noise-free condition, the iterated error of the APA-REE algorithm is almost only caused by the iterated direction {𝝋_𝑛}. The stability for the APA-REE algorithm has also been analyzed and shown that the range of the step size 𝜇<2(𝑁−𝑚−2) is much wider than the step size range 𝜇<2 of the APA algorithm given by [6]. And the steady-state weights of the APA-REE algorithm are proved to be unbiasedness and consistency. The simulation results show that the APA-REE algorithm improves the adaptive filtering convergence rate compared with the APA algorithm.

Acknowledgment

This paper was sponsored in part by the Basic Research Foundation of Northwestern Polytechnical University under Grant JC20100217.