#### Abstract

We address the performance analysis of the maximum likelihood (ML) direction-of-arrival (DOA) estimation algorithm in the case of azimuth/elevation estimation of two incident signals using the uniform circular array (UCA). Based on the Taylor series expansion and approximation, we get explicit expressions of the root mean square errors (RMSEs) of the azimuths and elevations. The validity of the derived expressions is shown by comparing the analytic results with the simulation results. The derivation in this paper is further verified by illustrating the consistency of the analytic results with the Cramer-Rao lower bound (CRLB).

#### 1. Introduction

There have been many studies on the direction-of-arrival (DOA) estimation [1–5]. Our concern in this paper is the performance analysis of the maximum likelihood- (ML-) based DOA estimation algorithm.

Many studies have been conducted on asymptotic performance analysis of maximum likelihood DOA estimation algorithm [6–8]. The shortcoming of the asymptotic performance analysis is that it is only applicable to high SNR region since it is based on approximation which is valid only for small amount of noise. To circumvent this problem, many studies on nonasymptotic performance analysis have been conducted for performance analysis which is valid for low SNR as well as high SNR.

In [9–12], the authors dealt with threshold performance analysis for single incident signal. On the other hand, our scheme presented in this paper is applicable to multiple incident signal as well as single incident signal.

In this paper, we address the performance analysis of the ML algorithm for the estimation of the DOA of incident signals. Our interest is in the case of multiple incident signals. The ML algorithm exploits the fact that, in the absence of the noise, the incident signal on the array elements can be expressed as a linear combination of the array steering vectors corresponding to the true incident angles.

Based on this observation, in the ML algorithm, the estimate of the DOA is obtained from the angles whose steering vectors can span the subspace to which the signal on the array elements belongs.

In practical situation, the noise is inevitable. Therefore, the noisy incident signal on the array elements cannot be expressed as a linear combination of the array steering vectors associated with true incident angles, which accounts for why estimation error occurs for noisy array response.

To quantify the bias of the estimate due to the noise on the antenna elements, we adopt the Taylor series approximation around the true incident angle. We derive the closed-form expression of the biases of the estimates, where different estimates are due to some approximations. For one of the many estimates, we derive the closed-form expression of the MSE of the estimate.

In Figure 1, the mathematical and statistical theories used for the performance analysis of the ML algorithm are enumerated.

Novelty of the proposed method over the existing methods is as follows.

*Performance Analysis for Multiple Incident Signals and Simultaneous Estimation of Azimuth/Elevation.* In [9–11], the authors made performance analysis of ML DOA algorithm for single incident signals. On the other hand, the scheme presented in this paper handles multiple incident signals as well as single incident signal. In addition, the studies presented in [9–11] consider the estimation of azimuth, not azimuth and elevation. Note that, in this paper, we consider the estimation of azimuth and elevation of multiple incident signals.

*Derivation of Explicit Expressions of Bias and MSE, Not the Lower Bound of MSE.* In [13], the authors derived the CRLB of the DOA algorithm, which is a lower bound of the variance of the DOA algorithm. Note that the CRLB is the lower bound of the variance, not the variance itself of the DOA algorithm. In this paper, explicit expression of the bias of the estimate and explicit expression of the MSE itself, not the CRLB, have been derived.

*Intuition on the Effect of Each Approximation on the Estimation Accuracy.* In this paper, we obtain, for , the expressions of [, ], [, ], and [, ]. The superscripts and denote approximation and approximation, respectively. Therefore, [, ], [, ], and [, ] are the estimates with no approximation, approximation, and approximations, respectively. By comparing these three values, we can intuitively recognize which approximation of approximation and approximation results in larger error. The importance of this observation is that the proposed scheme gives us an intuition on which approximation of all the approximations results in large approximation error. The approach presented in [14] does not show any intuition on which approximation results in large approximation error, since in [14] the authors only explicitly consider Taylor series expansion for general problem. Since the approach in [14] is a general approach applicable to many estimation problems, it does not give explicit expressions specific to DOA estimation problem. Since the scheme presented in this paper only considers the estimation of DOA estimation, rather than general estimation problem, the expressions presented in this paper are more explicit than the results presented in [14].

*Explicit Expressions of Various Estimates for Each Step of Successive Approximations.* To the best of our knowledge, there has been no study where explicit expressions of successive estimates have been derived [6–13]. Successive estimates imply various estimates obtained as each approximation has been successively applied. In [14], no explicit expression of estimate for each step of successive approximations has been presented. Although the study presented in [14] deals with the estimation of vector parameter as well as scalar parameter, the results in [14] for vector parameter estimation, equations and in [14], are quite implicit, rather than explicit, and it is very difficult to get intuition on how the bias and the covariance matrix are dependent on the amount of an additive noise.

*Search-Free Approximate ML DOA Estimation and Derivation of Closed-Form Estimation Error.* For getting and in (13) for , four-dimensional search with respect to , , , and has to be performed. In general, for estimation of incident signals, dimensional search has to be performed for simultaneous estimation of , which is computationally very intensive. In addition, in practical implementation of (13), the estimates are highly dependent on the search range and search step, which implies that the estimates in (13) may be wrong if search step and search range are not properly chosen. In summary, the original ML estimate obtained from implementation of (13) has two demerits of computational cost and dependence on search step and search range.

On the other hand, and are obtained from (42) and (45), respectively. Note that and are obtained in closed-form and that evaluation of and is search-free, which implies that evaluation of and is much less computationally intensive than evaluation of . In addition, since evaluation of and is search-free, ] and are not dependent on search parameters such as search step and search range.

[] and [] in (42) and (45) can be regarded as estimation errors for approximate ML DOA algorithm. [] and [] can be obtained in closed-form, although [] and [] are less accurate than [] in the sense that approximation is applied in getting [] and that approximations are applied in getting [].

*Derivation of Analytic Expression of Closed-Form MSE of **.* Many evaluations of (42) and (45) are required for getting empirical performance analysis. Single evaluation of (42) and (45) is not computationally intensive since they are search-free closed-form solutions. Note that main computational cost of evaluating (42) and (45) is inversion of matrices, which is not computationally intensive at all. For estimation of incident signals, matrices have to be inverted to implement (42) and (45). But for empirical performance analysis based on Monte-Carlo simulation, (42) and (45) have to be evaluated many times. Note that the number of repetitions should be large for getting reliable empirical performance analysis, which implies that the empirical performance analysis based on many evaluations of (42) and (45) is computationally intensive. In this paper, it has been shown that empirical performance of [, ] can be obtained from (47), without many evaluations of (45). Note that single evaluation of (47) results in analytic performance analysis, which should be equivalent to empirical performance based on many repetitions of (45). Therefore, although it is not necessary to repeatedly evaluate (45) to get empirical MSE, (45) is evaluated many times for validating the derived expression (47). Also, note that since analytic expression of the MSE of [, ] is not available, the MSE of [, ] has to be empirically obtained via many evaluations of (42) with different realizations of additive noise for each evaluation.

In and , the superscripts and denote approximation and approximation, respectively. On the other hand, in [14], no explicit expression of MSE is presented, and only the implicit expression of final estimate is presented. Note that, in this paper, explicit expressions of all the estimates associated with each successive approximation have been presented.

#### 2. ML DOA Algorithm

For simplicity, we present performance analysis for two incident signals on the UCA in this paper. But, extension to more incident signals and general planar array structure is straightforward. Let , , , and representThe number of the antenna elements is denoted by . When there is no noise, the received signals on the antenna elements are expressed aswhere is given byArray matrix in (2) is written aswhere array vector is defined as

The corresponding expression for the noisy signals iswhere and denote the azimuth and the elevation, respectively.

Expression (7) for a general planar array with elements can be written aswhere and denote the element spacing in the -direction and -direction, respectively, and and denote the index in the -direction and -direction, respectively.

Also, (8) for general planar array is written asWe have explicitly shown how (7) and (8) should be modified for a two-dimensional planar array. Note that, for an arbitrary array structure, (7) and (8) are modified appropriately to apply the proposed algorithm to the specific array structure. (7) for an arbitrary array structure can be easily obtained from the phase difference between antenna elements, given the azimuth and the elevation of the th incident signal. Note that the phase difference is both dependent on the array geometry for a specific array structure and the direction of the incident signal.

The projection matrix onto the column space of can be expressed as [15]. It is assumed that the entries of the Gaussian vector are independent and identically distributed Gaussian random variables with zero-mean and the same variance. Note that the noise is complex-valued and that the real part and the imaginary part of the noise are independent Gaussian random variables with zero-mean. The variance of the real part is denoted by , which is equal to the variance of the imaginary part, :Note that, in this paper, the formulation is for the case where there are two incident signals. But, it can be easily extended to the case of more incident signals. Using the maximum likelihood (ML) algorithm, the estimates are given by [16]More specifically, (12) for is written asFor notational simplicity, is used to denote : is defined fromThe explicit expression of the entries of the matrix in (11) is given byUsing (4) and (5), in (16) is given byLet represent :

Using (16) and (17), can be expressed as

Using (17), can be written as

The explicit expression of the entries of the matrix in (11) is given by

Using (16)–(21), in (22) is defined as

Note that denotes the entry at the th row and the th column of . The explicit expression of the entries of the matrix in (12) is given by

Using (24) in (12), we get . The biases of the estimates are defined as

#### 3. Closed-Form Expression of Estimation Error

Since , , , and should be the maximizing arguments of , the first-order derivative of with respect to each argument should be zero at where the first-order partial derivatives of in (26) with respect to each argument are expressed as, using (22),The first-order partial derivatives of in (20) and in (23) with respect to each argument in (27) are expressed in the Appendices A and B. Denominator of equation (27) can be written asUsing (21), is defined asWe define in (26) aswhere is given in (2). Let represent the difference between and :Substituting (32) in (26) and applying the first-order Taylor expansion of , , , and yieldLet , , , , , and be defined aswhere denotes the th component of and and denote the th component of and , respectively. Using (34)–(36), (33) can be expressed in matrix formwhere is defined asNote that in (34) denotes the entry at the th row and the th column of in (38), not at the th row and the th column. should be maximized at , since, with the sample covariance matrix from noiseless response, the ML algorithm estimates the DOA accurately. Therefore, the partial derivatives of with respect to the each entry of should be zero at The last equalities of (39) can be expressed asSubstituting (40) in (35), is simplified to Using (38) and (35), in (37) is defined asSince is true in (34), (34) can be approximated aswhere the superscript denotes that the approximation based on is applied. Accordingly, is defined asThe first-order derivatives of , and in (34) are derived in Appendix G. Applying -approximation to (37), is given bywhere an explicit expression of the entries of is

#### 4. Closed-Form Expression of Mean Square Error

Using (45), an analytic MSE of can be expressed asThe entry at the th row and the th column of can be expressed as

in (48) is derived in Appendix P. Entries of in (47) are written asFrom (49), it is clear that the MSEs of , , , and are given bywhere and are true since and are real-valued. Mathematical details on the derivation are summarized as follows.

*Mathematical Details on the Performance Analysis of ML DOA Estimation.* Number of snapshots is , number of arrays is , and number of signals is 2.Projection matrix of The derivatives of the ML cost function, based on the covariance matrix associated with noisy array response, with respect to each incident signal, are not identically zero at true incident angles. From the angles at which the derivative is identically zero, the expression of the estimation error can be derivedThe derivative of the ML cost function, based on the covariance matrix associated with noiseless array response, with respect to each incident signal, is identically zero at true incident anglesSubstituting (55) and (56) in (58) results in

Using in (59) results inAn analytic expression of