Experimental and Analytical Studies on Improved Feedforward ML Estimation Based on LS-SVR

Abstract

Maximum likelihood (ML) algorithm is the most common and effective parameter estimation method. However, when dealing with small sample and low signal-to-noise ratio (SNR), threshold effects are resulted and estimation performance degrades greatly. It is proved that support vector machine (SVM) is suitable for small sample. Consequently, we employ the linear relationship between least squares support vector regression (LS-SVR)’s inputs and outputs and regard LS-SVR process as a time-varying linear filter to increase input SNR of received signals and decrease the threshold value of mean square error (MSE) curve. Furthermore, it is verified that by taking single-tone sinusoidal frequency estimation, for example, and integrating data analysis and experimental validation, if LS-SVR’s parameters are set appropriately, not only can the LS-SVR process ensure the single-tone sinusoid and additive white Gaussian noise (AWGN) channel characteristics of original signals well, but it can also improves the frequency estimation performance. During experimental simulations, LS-SVR process is applied to two common and representative single-tone sinusoidal ML frequency estimation algorithms, the DFT-based frequency-domain periodogram (FDP) and phase-based Kay ones. And the threshold values of their MSE curves are decreased by 0.3 dB and 1.2 dB, respectively, which obviously exhibit the advantage of the proposed algorithm.

1. Introduction

Maximum likelihood (ML) estimation depends on the asymptotic theory, which means that the statistical characteristics are shown accurately only when the sample size is infinity. However, burst-mode transmissions always bring problems about short data and severe conditions. Therefore, threshold effect is existing. Namely, the mean square error (MSE) of ML estimation can reach Cramer-Rao lower bound (CRLB) if it is higher than a value, or the performance will be deteriorated rapidly.

Statistical learning theory (SLT) and structure risk minimization (SRM) rule in it are specialized in small-sample learning [1]. As their concrete implement, support vector machine (SVM) overcomes the over-fitting and local minimum problems currently existing in artificial neural network (ANN). Least squares support vector regression (LS-SVR) has the following improvements: inequality constraint are substituted by equality one; a squared loss function is taken for the error variable. Hence, we introduce LS-SVR to improve ML estimator and take feedforward single-tone sinusoidal frequency estimation for example, in this study.

Estimating frequency of a single-tone sinusoid has attracted considerable attention for many decades. Rife and Boorstyn exploited the relationship of maximum likelihood estimator (MLE) to discrete Fourier transform (DFT) and proposed a frequency-domain periodogram (FDP) algorithm having two stages: coarse search and fine search [2]. In order to reduce the calculation cost, a great deal of improved algorithms have erupted mainly from two sides: interpolation-based and phase-based ones.

During the former, an iterative binary search for the true signal frequency has been presented, which is particularly suited for digital signal processing (DSP) implementation [3]. In [4], the same authors have proposed a number of hybrid estimators that combine the dichotomous search with various interpolation techniques in order to reduce the computational complexity, at the expense of acquisition range. And, other modified dichotomous search frequency estimators have been addressed in [5–7]. Besides, complex Fourier coefficients have been utilized to interpolate the true signal frequency between the maximum and the second highest bin [8]. However, it has been shown to have a frequency dependent performance [9]. Two improved estimators have been proposed, which were implemented iteratively [10, 11]. Rational combination of three spectrum lines (RCTSL) has been employed as the fine estimation, because of its constant combinational weights in least square approximation [12]. Other methods used for interpolation include Lagrange interpolator [13], L-filter DFT [14], nonlinear filter [15], Kaise window [16], trigonometric polynomial interpolator [17], narrowband approximation interpolator [18], and so on. In the latter, Tretter [19] was the first person to propose a phase-based approach by introducing an approximated and linear model for instantaneous signal phase. Subsequently, a great deal of improvements have erupted mainly in the following three parts: taking differences over one or more delays, which is well-known as Kay and generalized Kay estimators [20–25]; introducing autocorrelations and their different functions, such as Fitz, L&R, and M&M estimators [26–30]; and preprocessing by means of lowpass filter, blocking average, and filter banks to increase signal-to-noise ratio (SNR) [31–34].

In this paper, we present an improved feedforward ML estimation based on LS-SVR, taking single-tone sinusoidal frequency estimation, for example. LS-SVR process is regarded as a time-varying linear filter to increase input SNR of received signals, and accordingly, the threshold value of MSE curve is decreased. Reliability and validation of LS-SVR process are verified by integrating data analysis and experimental simulation. It is verified that not only can the LS-SVR ensure the single-tone sinusoid and AWGN channel characteristics of original signals well, but also increases the input SNR of received signals efficiently and improves frequency estimation performance. During experimental simulations, LS-SVR process is applied to two common and representative single-tone sinusoidal frequency estimation algorithms, the DFT-based FDP and phase-based Kay algorithms. The estimation performance of having the LS-SVR process and not are compared, respectively, to exhibit the advantage of the proposed algorithm, if its parameters are set appropriately.

The remainder of this paper is organized as follows. Section 2 briefly introduces the basic theory of LS-SVR. Section 3 describes the model of single-tone sinusoidal frequency estimation and the classical algorithms including FDP and Kay. In Section 4, the LS-SVR process is concretely explained and analyzed. And Section 5 shows the results of simulations and experiments. The paper is concluded in Section 6 finally.

2. Theory of LS-SVR

At first, a linear hyperplane 𝑓(𝐱)=(𝐰⋅𝜙(𝐱))+𝑏 insofar as for 𝜀 is assumed to fit all elements of the training set 𝑆={(𝐱₁,𝑦₁),…,(𝐱_𝑁,𝑦_𝑁)}⊂𝑅^𝑛×𝑅, where 𝐰 is the high-dimensional coefficient of 𝑓(𝐱), (⋅) is an inner product operator, and 𝜙(⋅) is a nonlinear mapping from low to high dimension feature space. Also, 𝜀-insensitive loss function is defined as 𝐿𝑦,𝑓(𝐱)=𝑦−𝑓(𝐱)_𝜀=(𝐱(𝐱0,𝑦−𝑓)≤𝜀𝑦−𝑓)−𝜀,else (1)

𝑑_𝑖 denotes the distance from point (𝐱_𝑖,𝑦_𝑖)∈𝑆 to 𝑓(𝐱): 𝑑_𝑖=𝐰⋅𝜙𝐱_𝑖+𝑏−𝑦_𝑖1+‖𝐰‖²≤𝜀1+‖𝐰‖²,𝑖=1,…,𝑁. (2)

According to (2), we optimize 𝑓(𝐱) through maximizing 𝜀/(1+‖𝐰‖²), that is, minimizing ‖𝐰‖². Thereby, SVR is presented as ()=1min𝐽𝐰,𝑏2‖𝐰‖²s.t.𝐰⋅𝜙𝐱_𝑖+𝑏−𝑦_𝑖≤𝜀,𝑖=1,…,𝑁. (3)

Then, we proceed to conquer inseparable condition by introducing error variables 𝑒_𝑖 and least squares (LS) method, and convert (3) into ()=1min𝐽𝐰,𝑏2‖𝐰‖²+𝐶2^𝑁_𝑖=1𝑒²_𝑖,s.t.𝑦_𝑖=𝐰⋅𝜙𝐱_𝑖+𝑏+𝑒_𝑖,𝑖=1,…,𝑁, (4) where penalty factor 𝐶 is a positive constant to take compromise in LS-SVR’s generalization capability and fitting errors, which are denoted by the first and second item of 𝐽(𝐰,𝑏), respectively.

Next step, we use Lagrange multiplier method and replace (𝜙(𝐱_𝑖)⋅𝜙(𝐱_𝑗)) with kernel function 𝐾(𝐱_𝑖,𝐱_𝑗): 0𝟏^𝑇1𝟏𝐐+𝐶𝐈𝑏𝜶=0𝐘, (5) where 𝟏=(1,…,1)^𝑇_𝑁, 𝐘=(𝑦₁,…,𝑦_𝑁)^𝑇, 𝜶=(𝛼₁,…,𝛼_𝑁)^𝑇, 𝛼₁,…,𝛼_𝑁 are Lagrange multipliers, 𝐐 is a kernel function matrix, and radius basis function (RBF) is adopted in this study, so: 𝐐_𝑖𝑗=𝐾𝐱_𝑖,𝐱_𝑗−𝐱=exp^𝑖−𝐱_𝑗²ℎ², (6) where 𝐐_𝑖𝑗 is the (𝑖,𝑗)th element of 𝐐; the width of RBF ℎ is a positive constant.

Ultimately, the discriminant function is described as 𝑓(𝐱)=𝐰⋅𝜙(𝐱)+𝑏=^𝑁_𝑖=1𝛼_𝑖𝐾𝐱_𝑖,𝐱+𝑏. (7)

3. Signal Model and Classical Algorithms

3.1. Signal Model

The sinusoid signal polluted by noise is modeled as 𝑟_𝑛=𝐴exp𝑗2𝜋𝑓𝑛+𝜃+𝑤_𝑛,𝑛=0,…,𝑁−1. (8) Here, 𝐴>0, 𝑓∈[−0.5,0.5), 𝜃∈[−𝜋,𝜋) are the amplitude, deterministic but unknown frequency, and initial phase, respectively; 𝑤_𝑛 is an independent complex additive white Gaussian noise (AWGN) with zero-mean and variance 𝜎²; and 𝑁 is the sample size.

3.2. Classical Algorithms

FDP algorithm extends the number of DFT points to 𝐾 by appending (𝐾−𝑁) zeros and searches the maximum bin [2]. Now, the frequency step is 1/𝐾. Consider 𝑓_FDP=argmax_𝑓𝑘𝑅𝑓_𝑘,𝑘=0,…,𝐾−1, (9) where 𝑅(𝑓_𝑘)=∑^𝑁−1_𝑛=0𝑟_𝑛𝑒^{−𝑗2𝜋𝑓𝑘𝑛}, 𝑓_𝑘=𝑘/𝐾. For the sake of convenience of fast Fourier transform (FFT) calculation, 𝐾 is usually set to 2𝑁,4𝑁,8𝑁,….

Kay algorithm measures differential phase with a delay of one symbol interval and derives the estimation value of frequency [20]. Consider 𝑓_Kay=12𝜋^𝑁−1_𝑘=1𝑤_𝑘arg𝑟_𝑘𝑟^∗_𝑘−1, (10) where the weights 𝑤_𝑘=6𝑘(𝑁−𝑘)/𝑁(𝑁²−1), 𝑘=1,…,𝑁−1.

4. LS-SVR Process and Its Analysis

4.1. LS-SVR Process

At first, we construct the training set 𝑆={(𝑥_𝑛,𝑦_𝑛)∣𝑛=1,…,𝑁}, where 𝑥_𝑛=𝑛−1 is time series, 𝑦_𝑛=𝑟_𝑛−1=𝐴exp{𝑗[2𝜋𝑓(𝑛−1)+𝜃]}+𝑤_𝑛−1=𝑔_𝑛+𝑤^′_𝑛 is received series, and 𝑔_𝑛 and 𝑤^′_𝑛 are signal and noise components of 𝑦_𝑛, perspectively.

Secondly, we try to fit 𝑆 by 𝑓(𝑥)=(𝐰⋅𝜙(𝑥))+𝑏, 𝑥∈[0,𝑁−1]. However, LS-SVR process is only available for real signal, so 𝑦_𝑛 is divided into real and image parts and preprocessed perspectively. That is,(1)we employ LS-SVR to fit training set 𝑆_real={(𝑥^real_𝑛,𝑦^real_𝑛)∣𝑛=1,…,𝑁}, where 𝑥^real_𝑛=𝑥_𝑛=𝑛−1, 𝑦^real_𝑛=Re[𝑦_𝑛]=𝐴cos[2𝜋𝑓(𝑛−1)+𝜃]+𝑤^real_𝑛−1, 𝑤^real_𝑛−1 is the real part of 𝑤_𝑛−1, so it is an independent real AWGN with zero-mean and variance 𝜎²/2, and derive 𝑓_real(𝑥),(2)we employ LS-SVR to fit training set 𝑆_imag={(𝑥^imag_𝑛,𝑦^imag_𝑛)∣𝑛=1,…,𝑁}, where 𝑥^imag_𝑛=𝑥_𝑛=𝑛−1, 𝑦^imag_𝑛=Im[𝑦_𝑛]=𝐴sin[2𝜋𝑓(𝑛−1)+𝜃]+𝑤^imag_𝑛−1, 𝑤^imag_𝑛−1 is the image part of 𝑤_𝑛−1, so it is an independent real AWGN with zero-mean and variance 𝜎²/2, and derive 𝑓_imag(𝑥),(3)𝑓(𝑥)=𝑓_real(𝑥)+𝑗×𝑓_imag(𝑥), 𝑥∈[0,𝑁−1].

Then, we substitute 𝑥_𝑛 into 𝑓(𝑥) and get a new series of received signals.

At last, we utilize the classical algorithm to estimate frequency accurately.

4.2. Analysis of LS-SVR Process

The real part of 𝑦_𝑛 is taken, for example, and now 𝐘=(𝑦^real₁,…,𝑦^real_𝑁)^𝑇=(𝐴cos𝜃+𝑤₀,…,𝐴cos[2𝜋𝑓(𝑁−1)+𝜃]+𝑤_𝑁−1)^𝑇. The solutions of (5) are described as 𝟏𝑏=^𝑇(𝐐+(1/𝐶)𝐈)⁻¹𝐘𝟏^𝑇(𝐐+(1/𝐶)𝐈)⁻¹𝟏1𝜶=𝐐+𝐶𝐈⁻¹(𝐘−𝟏𝑏). (11)

𝐘=(𝑦₁,…,𝑦_𝑁)^𝑇=(𝑓(𝑥₁),…,𝑓(𝑥_𝑁))^𝑇 is defined as outputs of LS-SVR process and (11) and (12) are substituted into (7) 1𝐘=𝐐𝜶+𝟏𝑏=𝐐𝐐+𝐶𝐈⁻¹𝐘+1𝐈−𝐐𝐐+𝐶𝐈⁻¹=𝐐1𝟏𝑏𝐐+𝐶𝐈⁻¹+1𝐈−𝐐𝐐+𝐶𝐈⁻¹×𝟏𝟏^𝑇(𝐐+(1/𝐶)𝐈)⁻¹𝟏^𝑇(𝐐+(1/𝐶)𝐈)⁻¹𝟏𝐘=𝜷𝐘. (12)

It is clear that now LS-SVR process can be regarded as a time-varying linear filter, and its coefficients are determined by 1𝜷=𝐐𝐐+𝐶𝐈⁻¹+1𝐈−𝐐𝐐+𝐶𝐈⁻¹×𝟏𝟏^𝑇(𝐐+(1/𝐶)𝐈)⁻¹𝟏^𝑇(𝐐+(1/𝐶)𝐈)⁻¹𝟏. (13)

Received signals polluted by noise are 𝐘=𝐆+𝐖, where 𝐆=(Re[𝑔₁],…,Re[𝑔_𝑁])^𝑇=(𝐴cos𝜃,…,𝐴cos[2𝜋𝑓(𝑁−1)+𝜃])^𝑇, and 𝐖=(Re[𝑤^′₁],…,Re[𝑤^′_𝑁])^𝑇=(𝑤^real₀,…,𝑤^real_𝑁−1)^𝑇 are signal and noise components of 𝐘, respectively. The outputs of LS-SVR process are 𝐘=𝜷𝐘=𝜷(𝐆+𝐖)=𝜷𝐆+𝜷𝐖=𝐆+𝐖, (14) where 𝐆=(𝑔₁,…,𝑔_𝑁)^𝑇, 𝐖=(𝑤₁,…,𝑤_𝑁)^𝑇 are the signal and noise components of 𝐘=(𝑦₁,…,𝑦_𝑁)^𝑇, respectively.

Further, the mean, autocorrelation, and covariance function of 𝐘 are obtained as follows, where 𝐸{⋅} is a mean operator: 𝐸𝐘{}𝐸𝐘𝐘=𝐸𝜷𝐆+𝜷𝐖=𝜷𝐆_𝑇=𝐸𝜷𝐘𝐘^𝑇𝜷^𝑇=𝜷𝐸𝐘𝐘^𝑇𝜷^𝑇=𝜷𝐸𝐆𝐆^𝑇+𝐸𝐖𝐖^𝑇𝜷^𝑇=𝜷𝐆𝐆^𝑇+𝜎²𝐈𝜷^𝑇, (15) where 𝐸{𝐘𝐘^𝑇}=𝐸{𝐆𝐆^𝑇}+𝐸{𝐖𝐖^𝑇}=𝐆𝐆^𝑇+𝜎²𝐈. Consider 𝐸𝐘−𝐸𝐘𝐘−𝐸𝐘^𝑇=𝐸𝐘𝐘^𝑇−𝐸𝐘𝐸𝐘^𝑇=𝜷𝐸𝐘𝐘^𝑇−𝐆𝐆^𝑇𝜷^𝑇=𝜎²𝜷𝜷^𝑇. (16)

Therefore, we conclude with the following.

(1) 𝑔_𝑛=∑^𝑁_𝑖=1𝛽_𝑛,𝑖𝑔_𝑖=𝐴∑^𝑁_𝑖=1𝛽_𝑛,𝑖cos[2𝜋𝑓(𝑖−1)+𝜃], 𝑛=1,…,𝑁, the signal components of outputs of LS-SVR process, do not have single-tone sinusoid characteristics, where 𝛽_𝑛,𝑖 is (𝑛,𝑖)th element of 𝜷. But 𝐆 will have the most powerful spetrum component in 𝑓 by selecting the width of RBF ℎ properly.

Firstly, setting 𝑓=0.15, 𝜃=0, 𝑁=32, SNR=0dB, and the parameter of LS-SVR 𝐶=5, the arbitrary amplitude spectrums of 𝐆, while ℎ=1 and ℎ=5 are illustrated in Figure 1, respectively. It is shown that when ℎ=1, the spectrum component of 𝐆 in 𝑓 is much more powerful than other places. It means that now the output of LS-SVR process still keeps the spectrum characteristics of 𝐴cos(2𝜋𝑓𝑥+𝜃) and can be used to estimate the frequency of 𝐴cos(2𝜋𝑓𝑥+𝜃). However, as ℎ increases, the spectrum component of 𝐆 in 𝑓 inversely decreases and others gradually increase; hence, now the output of LS-SVR process cannot keep the spectrum characteristics of 𝐴cos(2𝜋𝑓𝑥+𝜃).

(a) ℎ=1

(b) ℎ=5

Figure 1 

Amplitude spectrums of 𝐆.

Furthermore, everything is as in Figure 1, while ℎ=1, ℎ=4, and ℎ=5, the time-domain waveforms of 𝐆 are plotted in Figure 2, perspectively. The conclusion of Figure 2 is consistent with Figure 1, just when ℎ=1, the amplitude of time-domain waveforms of 𝐆 is less than 𝐴cos(2𝜋𝑓𝑥+𝜃).

(a) Original signals

(b) ℎ=1

(c) ℎ=4

(d) ℎ=5

Figure 2 

Time-domain waveforms of 𝐆.

The Euclidean distance between 𝐆 and 𝐆 is defined as follows, where max(⋅) is the operation of taking maximum value: 𝑑=^𝑁_𝑛=1𝐸𝑔_𝑛max𝐸𝑔_𝑛−𝑔_𝑛max𝑔_𝑛². (17)

Everything is as in Figure 1, and the number of Monte Carlo experiments is 10000; the values of 𝑑 with different ℎ are listed in Table 1. Obviously, 𝐆 can be very close to 𝐆 through proper choice of ℎ. Nevertheless, 𝐆 will gradually deviate from 𝐆as ℎ increases.

Table 1 

Values of 𝑑 with different ℎ.

Consequently, proper choice of ℎ ensures that LS-SVR process can be used for frequency estimation of single-tone sinusoidal signals. Integrating the above analyses, the value of ℎ must be less than 3.

(2) Through analyzing the covariance function of 𝐘, it is shown that LS-SVR process is feasible and valid with a proper choice of 𝐶 and ℎ.

From (16), it is obvious that the covariance function is related to 𝜎² and 𝜷. Taking 𝑁=4, for example, when 𝐶=5, ℎ=1, (16) is calculated as 𝜎²⋅0.868780.084560.005550.041120.084560.827230.082670.005550.005550.082670.827230.084550.041120.005550.084550.86878. (18)

We can deduce the following by analyzing (18)(A)The elements except main diagonal ones denote the correlations between 𝑤_𝑛 in different moments. Compared with the main diagonal elements, these elements are rather small. Hereby, 𝑤_𝑛 in different moments are approximately uncorrelated with each other. In addition, they are certainly subjected to Gaussian distribution, as the reason that LS-SVR process is linear. As result, 𝑤_𝑛 in different moments are almost independent.(B)The main diagonal elements denote the powers of 𝑤_𝑛. 𝑤_𝑛 in different moments are independent and identically distributed (i.i.d) by reason of their nearly equal values. And also, it is the premise that the classical algorithms of feedforward ML frequency estimation can be still employed after LS-SVR process.(C)Integrating (12), when 𝐶 decreases rapidly, 𝐐+(1/𝐶)𝐈≈(1/𝐶)𝐈, 𝜷≈(𝟏𝟏^𝑇/𝑁). Clearly, now 𝑦_𝑛 in different moments are equal and can not fit 𝐘 correctly, which means LS-SVR process is ineffective. On the contrary, when 𝐶 increases and tends to infinity, now 𝐐+(1/𝐶)𝐈≈𝐐, 𝜷≈𝐈, which means that LS-SVR process can hardly influence 𝐘. We apply LS-SVR process to FDP algorithm and derive the proposed algorithm called LS-SVR for short. Everything is as in Figure 1, and the number of Monte Carlo experiments is 10000; Figure 3 illustrates the impact of 𝐶 on MSE performance, which is consistent with all analyses above, also 𝐶=5 is set in this study.

Figure 3 

Impact of 𝐶 on MSE performance.

At the same time, according to𝐐_𝑖𝑗=𝐾𝑥_𝑖,𝑥_𝑗−=exp𝑖−𝑗²ℎ²,lim_ℎ→0𝐐_𝑖𝑗=1,𝑖=𝑗0,𝑖≠𝑗, (19) thus, lim_ℎ→0𝐐=𝐈. Integrating (12), lim_ℎ→0𝜷=(𝐶/(𝐶+1))𝐈+(1/(𝐶+1))(𝟏_𝑁𝟏^𝑇_𝑁)/𝑁≈(𝐶/(𝐶+1))𝐈, while 𝐶=5, 𝑁=32, which means that LS-SVR process can influence 𝐘 in a fixed proportion and barely improves the frequency estimation performance. On the contrary, lim_ℎ→+∞𝑄_𝑖𝑗=1, 𝑖≠𝑗. As a result, the correlations between 𝑤_𝑛 in different moments rise and their powers are not equal. When 𝐶=5, ℎ=5, (16) is calculated as 𝜎²⋅0.427480.311880.185280.075360.311880.274180.228660.185280.185280.228660.274180.311880.075360.185280.311880.42748. (20)

Everything is as in Figure 3 other than that 𝐶=5; the impact of ℎ on MSE performance is shown in Figure 4, which is consistent with all analyses above, and ℎ=1 is set in this study.

Figure 4 

Impact of ℎ on MSE performance.

(3) Setting 𝐶 and ℎ appropriately, LS-SVR process can increase SNR of 𝐘 and improve the performance of feedforward ML frequency estimations under the condition of small sample and low SNR.

We assume 𝜆_in=(1/𝐿𝑃)𝐆^𝑇𝐆/(1/𝐿𝑃)𝐖^𝑇𝐖=((𝐆^𝑇𝐆)/(𝐖^𝑇𝐖)), 𝜆_out=(((1/𝐿𝑃)𝐆^𝑇𝐆)/((1/𝐿𝑃)𝐖^𝑇𝐖))=((𝐆^𝑇𝐆)/(𝐖^𝑇𝐖)), and show the SNR variation between before and after LS-SVR process by 𝜆_out𝜆_in=𝐆^𝑇𝐆𝐖^𝑇𝐖𝐖^𝑇𝐖𝐆^𝑇𝐆=𝐆^𝑇𝜷^𝑇𝜷𝐆𝐖^𝑇𝐖𝐖^𝑇𝜷^𝑇𝜷𝐖𝐆^𝑇𝐆. (21)

10lg(𝐸{𝜆_out/𝜆_in}) is defined as the plus of LS-SVR process, where 𝜆_out/𝜆_in is related to 𝜷 which is determined by 𝐶 and ℎ. Everything is still as in Figure 1, the pluses of LS-SVR process with different 𝐶 and ℎ are listed in Table 2, which are consistent with Figures 3 and 4.

Table 2 

Pluses of LS-SVR process with different 𝐶 and ℎ.

5. Simulations and Experiments

We apply LS-SVR process to two common and representative single-tone sinusoidal ML frequency estimation algorithms, the DFT-based FDP and phase-based Kay ones, and derive the proposed algorithm called LS-SVR for short, where the number of DFT points of FDP algorithm is 𝐾=32𝑁.

5.1. Mean Performance

Everything is as in Figure 3 other than 𝐶=5; Figures 5 and 6 illustrate the mean of these three algorithms with different SNR. As is shown, whether high or low SNR, LS-SVR process can hardly change the unbiased ranges of FDP and Kay algorithms. Also, the unbiased ranges of all three algorithms will degrade with deterioration of SNR.

(a) SNR is −4 dB

(b) SNR is 0 dB

Figure 5 

Mean of FDP and LS-SVR algorithms.

(a) SNR is 6 dB

(b) SNR is 10 dB

Figure 6 

Mean of Kay and LS-SVR algorithms.

5.2. MSE Performance

Everything is as in Figure 3 except 𝐶=5; the MSE curves of these three algorithms are both shown in Figures 7 and 8, where CRLB is defined as (1/SNR)(3/(2𝜋)²𝑁(𝑁−1)(2𝑁−1)) [2]. We can see that LS-SVR process effectively improves the MSE performance of both FDP and Kay algorithm, and their threshold values are decreased by 0.3 dB and 1.2 dB, respectively.

(a) SNR∈[−6,3]dB

(b) SNR∈[−2,−1]dB

Figure 7 

MSE curves of FDP and LS-SVR algorithms.

(a) SNR∈[4,13]dB

(b) SNR∈[8,10]dB

Figure 8 

MSE curves of Kay and LS-SVR algorithms.

5.3. Impact of Sample Size 𝑁

Everything is still as in Figure 3 except that 𝐶=5, Figure 9 illustrates the impact of 𝑁 on MSE performance. We can know that the MSE curve of LS-SVR algorithm will decrease as 𝑁 increases. However, when LS-SVR process is applied to FDP algorithm, its threshold value will increase as 𝑁 increases; when LS-SVR process is applied to Kay algorithm, its threshold value will keep the same. The reason is related to the concrete frequency estimation algorithm after LS-SVR process.

(a) FDP and LS-SVR algorithms

(b) Kay and LS-SVR algorithms

Figure 9 

Impact of 𝑁 on MSE performance.

6. Conclusions

LS-SVR, as a modified and expanded version of SVR, still keeps its good capabilities in generalizing, high-dimensional processing, and nonlinear processing. Moreover, the equality constraint and squared loss function of LS-SVR make its computation and analysis rather simple.

In this paper, we take single-tone sinusoidal frequency estimation, for example, and propose an improved feedforward ML algorithm for NDA estimation based on LS-SVR. Also, we demonstrate its feasibility and validity by statistical analyses and experimental simulations, and discuss the proper parameter settings of LS-SVR. Our results suggest that the threshold value of MSE curve of the proposed algorithm is lower than the original one. More importantly, the analyses and applications presented in this paper can be used in other fields of parameter estimation in the same way, which provide a novel thought to the corresponding research.

The problems waiting for solution and the next direction are generalized as follows:(1)increasing LS-SVR’s operating efficiency and improving the real-time characteristics of parameter estimation based on LS-SVR,(2)making the best choice of LS-SVR’s parameters: the penalty factor 𝐶 and the width of RBF ℎ, not only the proper setting.

Acknowlegdment

This work was supported by the Open Research Project and Foundation of Complex Electromagnetic Environment Effects on Electronics and Information System under Grant no. CEMEE2014K0210B.