Filtering and Control for Unreliable Communication: The DiscreteTime Case
View this Special IssueResearch Article  Open Access
Experimental and Analytical Studies on Improved Feedforward ML Estimation Based on LSSVR
Abstract
Maximum likelihood (ML) algorithm is the most common and effective parameter estimation method. However, when dealing with small sample and low signaltonoise ratio (SNR), threshold effects are resulted and estimation performance degrades greatly. It is proved that support vector machine (SVM) is suitable for small sample. Consequently, we employ the linear relationship between least squares support vector regression (LSSVR)’s inputs and outputs and regard LSSVR process as a timevarying linear filter to increase input SNR of received signals and decrease the threshold value of mean square error (MSE) curve. Furthermore, it is verified that by taking singletone sinusoidal frequency estimation, for example, and integrating data analysis and experimental validation, if LSSVR’s parameters are set appropriately, not only can the LSSVR process ensure the singletone sinusoid and additive white Gaussian noise (AWGN) channel characteristics of original signals well, but it can also improves the frequency estimation performance. During experimental simulations, LSSVR process is applied to two common and representative singletone sinusoidal ML frequency estimation algorithms, the DFTbased frequencydomain periodogram (FDP) and phasebased Kay ones. And the threshold values of their MSE curves are decreased by 0.3 dB and 1.2 dB, respectively, which obviously exhibit the advantage of the proposed algorithm.
1. Introduction
Maximum likelihood (ML) estimation depends on the asymptotic theory, which means that the statistical characteristics are shown accurately only when the sample size is infinity. However, burstmode transmissions always bring problems about short data and severe conditions. Therefore, threshold effect is existing. Namely, the mean square error (MSE) of ML estimation can reach CramerRao lower bound (CRLB) if it is higher than a value, or the performance will be deteriorated rapidly.
Statistical learning theory (SLT) and structure risk minimization (SRM) rule in it are specialized in smallsample learning [1]. As their concrete implement, support vector machine (SVM) overcomes the overfitting and local minimum problems currently existing in artificial neural network (ANN). Least squares support vector regression (LSSVR) has the following improvements: inequality constraint are substituted by equality one; a squared loss function is taken for the error variable. Hence, we introduce LSSVR to improve ML estimator and take feedforward singletone sinusoidal frequency estimation for example, in this study.
Estimating frequency of a singletone sinusoid has attracted considerable attention for many decades. Rife and Boorstyn exploited the relationship of maximum likelihood estimator (MLE) to discrete Fourier transform (DFT) and proposed a frequencydomain periodogram (FDP) algorithm having two stages: coarse search and fine search [2]. In order to reduce the calculation cost, a great deal of improved algorithms have erupted mainly from two sides: interpolationbased and phasebased ones.
During the former, an iterative binary search for the true signal frequency has been presented, which is particularly suited for digital signal processing (DSP) implementation [3]. In [4], the same authors have proposed a number of hybrid estimators that combine the dichotomous search with various interpolation techniques in order to reduce the computational complexity, at the expense of acquisition range. And, other modified dichotomous search frequency estimators have been addressed in [5–7]. Besides, complex Fourier coefficients have been utilized to interpolate the true signal frequency between the maximum and the second highest bin [8]. However, it has been shown to have a frequency dependent performance [9]. Two improved estimators have been proposed, which were implemented iteratively [10, 11]. Rational combination of three spectrum lines (RCTSL) has been employed as the fine estimation, because of its constant combinational weights in least square approximation [12]. Other methods used for interpolation include Lagrange interpolator [13], Lfilter DFT [14], nonlinear filter [15], Kaise window [16], trigonometric polynomial interpolator [17], narrowband approximation interpolator [18], and so on. In the latter, Tretter [19] was the first person to propose a phasebased approach by introducing an approximated and linear model for instantaneous signal phase. Subsequently, a great deal of improvements have erupted mainly in the following three parts: taking differences over one or more delays, which is wellknown as Kay and generalized Kay estimators [20–25]; introducing autocorrelations and their different functions, such as Fitz, L&R, and M&M estimators [26–30]; and preprocessing by means of lowpass filter, blocking average, and filter banks to increase signaltonoise ratio (SNR) [31–34].
In this paper, we present an improved feedforward ML estimation based on LSSVR, taking singletone sinusoidal frequency estimation, for example. LSSVR process is regarded as a timevarying linear filter to increase input SNR of received signals, and accordingly, the threshold value of MSE curve is decreased. Reliability and validation of LSSVR process are verified by integrating data analysis and experimental simulation. It is verified that not only can the LSSVR ensure the singletone sinusoid and AWGN channel characteristics of original signals well, but also increases the input SNR of received signals efficiently and improves frequency estimation performance. During experimental simulations, LSSVR process is applied to two common and representative singletone sinusoidal frequency estimation algorithms, the DFTbased FDP and phasebased Kay algorithms. The estimation performance of having the LSSVR process and not are compared, respectively, to exhibit the advantage of the proposed algorithm, if its parameters are set appropriately.
The remainder of this paper is organized as follows. Section 2 briefly introduces the basic theory of LSSVR. Section 3 describes the model of singletone sinusoidal frequency estimation and the classical algorithms including FDP and Kay. In Section 4, the LSSVR process is concretely explained and analyzed. And Section 5 shows the results of simulations and experiments. The paper is concluded in Section 6 finally.
2. Theory of LSSVR
At first, a linear hyperplane insofar as for is assumed to fit all elements of the training set , where is the highdimensional coefficient of , is an inner product operator, and is a nonlinear mapping from low to high dimension feature space. Also, insensitive loss function is defined as
denotes the distance from point to :
According to (2), we optimize through maximizing , that is, minimizing . Thereby, SVR is presented as
Then, we proceed to conquer inseparable condition by introducing error variables and least squares (LS) method, and convert (3) into where penalty factor is a positive constant to take compromise in LSSVR’s generalization capability and fitting errors, which are denoted by the first and second item of , respectively.
Next step, we use Lagrange multiplier method and replace with kernel function : where , , , are Lagrange multipliers, is a kernel function matrix, and radius basis function (RBF) is adopted in this study, so: where is the th element of ; the width of RBF is a positive constant.
Ultimately, the discriminant function is described as
3. Signal Model and Classical Algorithms
3.1. Signal Model
The sinusoid signal polluted by noise is modeled as Here, , , are the amplitude, deterministic but unknown frequency, and initial phase, respectively; is an independent complex additive white Gaussian noise (AWGN) with zeromean and variance ; and is the sample size.
3.2. Classical Algorithms
FDP algorithm extends the number of DFT points to by appending zeros and searches the maximum bin [2]. Now, the frequency step is . Consider where , . For the sake of convenience of fast Fourier transform (FFT) calculation, is usually set to .
Kay algorithm measures differential phase with a delay of one symbol interval and derives the estimation value of frequency [20]. Consider where the weights , .
4. LSSVR Process and Its Analysis
4.1. LSSVR Process
At first, we construct the training set , where is time series, is received series, and and are signal and noise components of , perspectively.
Secondly, we try to fit by , . However, LSSVR process is only available for real signal, so is divided into real and image parts and preprocessed perspectively. That is,(1)we employ LSSVR to fit training set , where , , is the real part of , so it is an independent real AWGN with zeromean and variance , and derive ,(2)we employ LSSVR to fit training set , where , , is the image part of , so it is an independent real AWGN with zeromean and variance , and derive ,(3), .
Then, we substitute into and get a new series of received signals.
At last, we utilize the classical algorithm to estimate frequency accurately.
4.2. Analysis of LSSVR Process
The real part of is taken, for example, and now . The solutions of (5) are described as
is defined as outputs of LSSVR process and (11) and (12) are substituted into (7)
It is clear that now LSSVR process can be regarded as a timevarying linear filter, and its coefficients are determined by
Received signals polluted by noise are , where , and are signal and noise components of , respectively. The outputs of LSSVR process are where , are the signal and noise components of , respectively.
Further, the mean, autocorrelation, and covariance function of are obtained as follows, where is a mean operator: where . Consider
Therefore, we conclude with the following.
(1) , , the signal components of outputs of LSSVR process, do not have singletone sinusoid characteristics, where is th element of . But will have the most powerful spetrum component in by selecting the width of RBF properly.
Firstly, setting , , , , and the parameter of LSSVR , the arbitrary amplitude spectrums of , while and are illustrated in Figure 1, respectively. It is shown that when , the spectrum component of in is much more powerful than other places. It means that now the output of LSSVR process still keeps the spectrum characteristics of and can be used to estimate the frequency of . However, as increases, the spectrum component of in inversely decreases and others gradually increase; hence, now the output of LSSVR process cannot keep the spectrum characteristics of .
(a)
(b)
Furthermore, everything is as in Figure 1, while , , and , the timedomain waveforms of are plotted in Figure 2, perspectively. The conclusion of Figure 2 is consistent with Figure 1, just when , the amplitude of timedomain waveforms of is less than .
(a) Original signals
(b)
(c)
(d)
The Euclidean distance between and is defined as follows, where is the operation of taking maximum value:
Everything is as in Figure 1, and the number of Monte Carlo experiments is 10000; the values of with different are listed in Table 1. Obviously, can be very close to through proper choice of . Nevertheless, will gradually deviate from as increases.

Consequently, proper choice of ensures that LSSVR process can be used for frequency estimation of singletone sinusoidal signals. Integrating the above analyses, the value of must be less than 3.
(2) Through analyzing the covariance function of , it is shown that LSSVR process is feasible and valid with a proper choice of and .
From (16), it is obvious that the covariance function is related to and . Taking , for example, when , , (16) is calculated as
We can deduce the following by analyzing (18)(A)The elements except main diagonal ones denote the correlations between in different moments. Compared with the main diagonal elements, these elements are rather small. Hereby, in different moments are approximately uncorrelated with each other. In addition, they are certainly subjected to Gaussian distribution, as the reason that LSSVR process is linear. As result, in different moments are almost independent.(B)The main diagonal elements denote the powers of . in different moments are independent and identically distributed (i.i.d) by reason of their nearly equal values. And also, it is the premise that the classical algorithms of feedforward ML frequency estimation can be still employed after LSSVR process.(C)Integrating (12), when decreases rapidly, , . Clearly, now in different moments are equal and can not fit correctly, which means LSSVR process is ineffective. On the contrary, when increases and tends to infinity, now , , which means that LSSVR process can hardly influence . We apply LSSVR process to FDP algorithm and derive the proposed algorithm called LSSVR for short. Everything is as in Figure 1, and the number of Monte Carlo experiments is 10000; Figure 3 illustrates the impact of on MSE performance, which is consistent with all analyses above, also is set in this study.
At the same time, according to thus, . Integrating (12), , while , , which means that LSSVR process can influence in a fixed proportion and barely improves the frequency estimation performance. On the contrary, , . As a result, the correlations between in different moments rise and their powers are not equal. When , , (16) is calculated as
Everything is as in Figure 3 other than that ; the impact of on MSE performance is shown in Figure 4, which is consistent with all analyses above, and is set in this study.
(3) Setting and appropriately, LSSVR process can increase SNR of and improve the performance of feedforward ML frequency estimations under the condition of small sample and low SNR.
We assume , , and show the SNR variation between before and after LSSVR process by
is defined as the plus of LSSVR process, where is related to which is determined by and . Everything is still as in Figure 1, the pluses of LSSVR process with different and are listed in Table 2, which are consistent with Figures 3 and 4.

5. Simulations and Experiments
We apply LSSVR process to two common and representative singletone sinusoidal ML frequency estimation algorithms, the DFTbased FDP and phasebased Kay ones, and derive the proposed algorithm called LSSVR for short, where the number of DFT points of FDP algorithm is .
5.1. Mean Performance
Everything is as in Figure 3 other than ; Figures 5 and 6 illustrate the mean of these three algorithms with different SNR. As is shown, whether high or low SNR, LSSVR process can hardly change the unbiased ranges of FDP and Kay algorithms. Also, the unbiased ranges of all three algorithms will degrade with deterioration of SNR.
(a) SNR is −4 dB
(b) SNR is 0 dB
(a) SNR is 6 dB
(b) SNR is 10 dB
5.2. MSE Performance
Everything is as in Figure 3 except ; the MSE curves of these three algorithms are both shown in Figures 7 and 8, where CRLB is defined as [2]. We can see that LSSVR process effectively improves the MSE performance of both FDP and Kay algorithm, and their threshold values are decreased by 0.3 dB and 1.2 dB, respectively.
(a)
(b)
(a)
(b)
5.3. Impact of Sample Size
Everything is still as in Figure 3 except that , Figure 9 illustrates the impact of on MSE performance. We can know that the MSE curve of LSSVR algorithm will decrease as increases. However, when LSSVR process is applied to FDP algorithm, its threshold value will increase as increases; when LSSVR process is applied to Kay algorithm, its threshold value will keep the same. The reason is related to the concrete frequency estimation algorithm after LSSVR process.
(a) FDP and LSSVR algorithms
(b) Kay and LSSVR algorithms
6. Conclusions
LSSVR, as a modified and expanded version of SVR, still keeps its good capabilities in generalizing, highdimensional processing, and nonlinear processing. Moreover, the equality constraint and squared loss function of LSSVR make its computation and analysis rather simple.
In this paper, we take singletone sinusoidal frequency estimation, for example, and propose an improved feedforward ML algorithm for NDA estimation based on LSSVR. Also, we demonstrate its feasibility and validity by statistical analyses and experimental simulations, and discuss the proper parameter settings of LSSVR. Our results suggest that the threshold value of MSE curve of the proposed algorithm is lower than the original one. More importantly, the analyses and applications presented in this paper can be used in other fields of parameter estimation in the same way, which provide a novel thought to the corresponding research.
The problems waiting for solution and the next direction are generalized as follows:(1)increasing LSSVR’s operating efficiency and improving the realtime characteristics of parameter estimation based on LSSVR,(2)making the best choice of LSSVR’s parameters: the penalty factor and the width of RBF , not only the proper setting.
Acknowlegdment
This work was supported by the Open Research Project and Foundation of Complex Electromagnetic Environment Effects on Electronics and Information System under Grant no. CEMEE2014K0210B.
References
 V. N. Vapnik, Statistical Learning Theory, John Wiley & Sons, New York, NY, USA, 1998.
 D. C. Rife and R. R. Boorstyn, “Singletone parameter estimation from discretetime observations,” IEEE Transactions on Information Theory, vol. 20, no. 5, pp. 591–598, 1974. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 Y. V. Zakharov and T. C. Tozer, “Frequency estimator with dichotomous search of periodogram peak,” Electronics Letters, vol. 35, no. 19, pp. 1608–1609, 1999. View at: Publisher Site  Google Scholar
 Y. V. Zakharov, V. M. Baronkin, and T. C. Tozer, “DFTbased frequency estimators with narrow acquisition range,” Proceedings of IEE Communications, vol. 148, no. 1, pp. 1–7, 2001. View at: Publisher Site  Google Scholar
 E. Aboutanios, “A modified dichotomous search frequency estimator,” IEEE Signal Processing Letters, vol. 11, no. 2, pp. 186–188, 2004. View at: Publisher Site  Google Scholar
 H. Xu and D. Zhang, “A simple iterative carrier frequency estimation algorithm,” in Proceedings of the International Conference on Networks Security, Wireless Communications and Trusted Computing (NSWCTC '09), pp. 724–727, Wuhan, China, April 2009. View at: Publisher Site  Google Scholar
 G.B. Zhang, “Novel algorithm for frequency estimation of sinusoid signal with random length,” in Proceedings of the International Conference on Electronic and Mechanical Engineering and Information Technology (EMEIT '11), pp. 523–526, Harbin, China, August 2011. View at: Publisher Site  Google Scholar
 B. G. Quinn, “Estimating frequency by interpolation using Fourier coefficients,” IEEE Transactions on Signal Processing, vol. 42, no. 5, pp. 1264–1268, 1994. View at: Publisher Site  Google Scholar
 B. G. Quinn, “Estimation of frequency, amplitude, and phase from the DFT of a time series,” IEEE Transactions on Signal Processing, vol. 45, no. 3, pp. 814–817, 1997. View at: Publisher Site  Google Scholar
 E. Aboutanios and B. Mulgrew, “Iterative frequency estimation by interpolation on Fourier coefficients,” IEEE Transactions on Signal Processing, vol. 53, no. 4, pp. 1237–1242, 2005. View at: Publisher Site  Google Scholar
 E. Aboutanios, “Generalised DFTbased estimators of the frequency of a complex exponential in noise,” in Proceedings of the 3rd International Congress on Image and Signal Processing (CISP '10), pp. 2998–3002, Yantai, China, October 2010. View at: Publisher Site  Google Scholar
 C. Yang and G. Wei, “A noniterative frequency estimator with rational combination of three spectrum lines,” IEEE Transactions on Signal Processing, vol. 59, no. 10, pp. 5065–5070, 2011. View at: Publisher Site  Google Scholar
 S. R. Dooley and A. K. Nandi, “Fast frequency estimation and tracking using Lagrange interpolation,” Electronics Letters, vol. 34, no. 20, pp. 1908–1910, 1998. View at: Publisher Site  Google Scholar
 I. Djurović and V. V. Lukin, “Estimation of singletone signal frequency by using the LDFT,” Signal Processing, vol. 87, no. 6, pp. 1537–1544, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 G. Campobello, G. Cannatá, N. Donato, A. Famulari, and S. Serrano, “A novel lowcomplex and lowmemory method for accurate singletone frequency estimation,” in Proceedings of the 4th International Symposium on Communications, Control, and Signal Processing (ISCCSP '10), pp. 1–6, Limassol, Cyprus, March 2010. View at: Publisher Site  Google Scholar
 S. W. Chen, D. H. Li, and X. P. Wei, “Accurate frequency estimation of real sinusoid signal,” in Proceedings of the 2nd International Conference on Signal Processing Systems, pp. 370–372, Dalian, China, July 2010. View at: Google Scholar
 Z. Ye, G. Xu, and D. Guo, “An accurate estimation algorithm of frequency and phase at low signalnoise ratio levels,” in Proceedings of the International Conference on Wireless Communications and Signal Processing (WCSP '10), pp. 1–5, Suzhou, China, October 2010. View at: Publisher Site  Google Scholar
 G. Wei, C. Yang, and F.J. Chen, “Closedform frequency estimator based on narrowband approximation under noisy environment,” Signal Processing, vol. 91, no. 4, pp. 841–851, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 S. A. Tretter, “Estimating the frequency of a noisy sinusoid by linear regression,” IEEE Transactions on Information Theory, vol. IT31, no. 6, pp. 832–835, 1985. View at: Google Scholar
 S. Kay, “A fast and accurate single frequency estimator,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, no. 12, pp. 1987–1990, 1989. View at: Publisher Site  Google Scholar
 V. Clarkson, P. J. Kootsookos, and B. G. Quinn, “Analysis of the variance threshold of Kay's weighted linear predictor frequency estimator,” IEEE Transactions on Signal Processing, vol. 42, no. 9, pp. 2370–2379, 1994. View at: Publisher Site  Google Scholar
 E. Rosnes and A. Vahlin, “Frequency estimation of a single complex sinusoid using a generalized Kay estimator,” IEEE Transactions on Communications, vol. 54, no. 3, pp. 407–415, 2006. View at: Publisher Site  Google Scholar
 H. C. So and F. K. W. Chan, “A generalized weighted linear predictor frequency estimation approach for a complex sinusoid,” IEEE Transactions on Signal Processing, vol. 54, no. 4, pp. 1304–1315, 2006. View at: Publisher Site  Google Scholar
 A. B. Awoseyila, C. Kasparis, and B. G. Evans, “Improved single frequency estimation with wide acquisition range,” Electronics Letters, vol. 44, no. 3, pp. 245–247, 2008. View at: Publisher Site  Google Scholar
 H. Fu and P. Y. Kam, “Improved weighted phase averager for frequency estimation of single sinusoid in noise,” Electronics Letters, vol. 44, no. 3, pp. 247–248, 2008. View at: Publisher Site  Google Scholar
 M. Luise and R. Reggiannini, “Carrier frequency recovery in alldigital modems for burstmode transmissions,” IEEE Transactions on Communications, vol. 43, no. 2–4, pp. 1169–1178, 1995. View at: Publisher Site  Google Scholar
 M. P. Fitz, “Further results in the fast estimation of a singletone frequency,” IEEE Transactions on Communications, vol. 42, no. 2–4, pp. 862–864, 1994. View at: Google Scholar
 U. Mengali and M. Morelli, “Dataaided frequency estimation for burst digital transmission,” IEEE Transactions on Communications, vol. 45, no. 1, pp. 23–25, 1997. View at: Publisher Site  Google Scholar
 S. H. Leung, Y. Xiong, and W. H. Lau, “Modified Kay's method with improved frequency estimation,” Electronics Letters, vol. 36, no. 10, pp. 918–920, 2000. View at: Publisher Site  Google Scholar
 K. W. K. Lui and H. C. So, “Twostage autocorrelation approach for accurate single sinusoidal frequency estimation,” Signal Processing, vol. 88, no. 7, pp. 1852–1857, 2008. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 D. Kim, M. J. Narasimha, and D. C. Cox, “An improved single frequency estimator,” IEEE Signal Processing Letters, vol. 3, no. 7, pp. 212–214, 1996. View at: Publisher Site  Google Scholar
 T. Brown and M. M. Wang, “An iterative algorithm for singlefrequency estimation,” IEEE Transactions on Signal Processing, vol. 50, no. 11, pp. 2671–2682, 2002. View at: Publisher Site  Google Scholar
 Y.C. Xiao, P. Wei, X.C. Xiao, and H.M. Tai, “Fast and accurate single frequency estimator,” Electronics Letters, vol. 40, no. 14, pp. 910–911, 2004. View at: Publisher Site  Google Scholar
 M. L. Fowler and J. Andrew Johnson, “Extending the threshold and frequency range for phasebased frequency estimation,” IEEE Transactions on Signal Processing, vol. 47, no. 10, pp. 2857–2863, 1999. View at: Publisher Site  Google Scholar
Copyright
Copyright © 2013 Xueqian Liu and Hongyi Yu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.