#### Abstract

An efficient technique of designing spatial matrix filter for array signal preprocessing based on convex programming was proposed. Five methods were considered for designing the filter. In design method 1, we minimized the passband fidelity subject to the controlled overall stopband attenuation level. In design method 2, the objective function and the constraint in the design method 1 were reversed. In design method 3, the optimal matrix filter which has the general mean square error was considered. In design method 4, the left stopband and the right stopband were constrained with specific attenuation level each, and the minimized passband fidelity was received. In design method 5, the optimization objective function was the sum of the left stopband and the right stopband attenuation levels with the weighting factors 1 and *γ*, respectively, and the passband fidelity was the constraints. The optimal solution of the optimizations above was derived by the Lagrange multiplier theory. The relations between the optimal solutions were analyzed. The generalized singular value decomposition was introduced to simplify the optimal solution of design methods 1 and 2 and enhanced the efficiency of solving the Lagrange multipliers. By simulations, it could be found that the proposed method was effective for designing the spatial matrix filter.

#### 1. Introduction

Spatial matrix filter can be used for array signal preprocessing in direction of arrival (DOA) estimation and matched field processing (MFP). The signals from the interested area are reserved and the interferences from other areas are restrained by preprocessing [1–3]. The estimation accuracy and the probability of resolution can be enhanced enormously. In addition, the use of spatial matrix filtering technology makes it possible to estimate bearings for more than narrow-band sources using an -element array [4]. In line array sonar signal processing, this method can be used to suppress the platform noise which depresses the capability of the sonar seriously and keeps it unresolved for a long time [5, 6].

The spatial matrix filter evolves from matrix filter which is more powerful for filtering short data records than the finite impulse response (FIR) digital filters [7–11]. In [8, 9], a semi-infinite optimization programming was constructed for filter design, which minimized the mean square error between the actual response and the desired response in the passband to ensure the stopband attenuation over continuously stopband frequency satisfied given specification. The matrix filter was used directly in DOA estimation of a linear array for improving the estimation accuracy. It is an approximation of the real spatial matrix filter in which the stopband attenuation over continuous directions satisfied the given specification. In [7], a convex programming is constructed by using least square or minimax criterion for matrix filter design; the responses over discrete frequency points are considered. The same criterion is used for designing spatial matrix filter with application to MFP in passive sonar [1]. Unfortunately, the optimal filter had not been given. As complex solving theory was needed, the filters designed by [1, 7] could not be used efficiently in DOA estimation either. In a recent paper [10, 11], the optimal matrix filters were obtained over continuous frequency point by using convex programming, and the optimal matrix filter for short data records processing was given. References [2, 3] put forward the concept of generalized spatial prefiltering, in which a second-order cone programming (SOCP) was constructed. The spatial matrix filter was used in DOA estimation [12] and MFP [2, 3]. A heavy computation load was required for a large number of sensors as the variable amount in the SOCP was at least the square number of the sensor amount. It was difficult to obtain the filter when the sensors increase to large number, and a long time was consumed.

In this paper, an effective technique of designing the spatial matrix filter based on convex programming for array signal preprocessing in DOA estimation was proposed. Five design methods were considered. All the optimal solutions were given directly by using the Lagrange multiplier theory. Theoretical analysis shows that the proposed methods are more efficient than the previous methods. Numerical results were presented to illustrate the effectiveness of the methods.

#### 2. Optimal Spatial Matrix Filter Design

Consider a uniform linear array with elements. Assume that there are narrow band signals with the same frequency incident onto the sensor array from directions . The received signals are given by where and are the -dimensional source signals; is the -dimensional noise; is the steering matrix, where is the steering vector, denotes the transpose of a matrix, and is the distance of neighbor sensors.

Assume that the passband, left stopband, and right stopband steering matrices are , , ,, and ,,, respectively, where , , and are the th, th, and th steering vectors of passband, left stopband, and right stopband. The stopband steering matrix . , , and are the discrete numbers of the passband, left stopband, and right stopband regions, respectively, , , , and for . is the number of stopband steering vectors.

The filtering operation can be expressed as where are the -dimensional output signals and is an spatial matrix filter. By preprocessing, the normalized passband fidelity and the normalized stopband attenuation level can be given by and , respectively. Similarly, the left stopband and the right stopband attenuation levels are and , where denotes the Frobenius norm of a matrix. The objective of a spatial matrix filter is to pass the interested signal over the passband and suppress the interference over the stopband. There are five methods to design the spatial matrix filter based on convex programming.

##### 2.1. Design Method 1 (DM1)

In this method, we minimize the normalized passband fidelity subject to the controlled normalized stopband attenuation constraint at the same time. In this optimization, the left stopband and the right stopband are treated as a whole: where defines the stopband attenuation level. The average stopband attenuation is decibels. When , then we will gain the average attenuation of dB on the stopband. The average passband fidelity level and the left and the right stopband attenuation levels are chosen with the same way as follows in DM2, DM4, and DM5.

##### 2.2. Design Method 2 (DM2)

In this method, the objective function and the constraint in the design method 1 are reversed: where defines the passband fidelity level.

##### 2.3. Design Method 3 (DM3)

The third method is to find out the optimal spatial matrix filter which has the general mean square error. In this method, the passband fidelity and stopband attenuation are treated exactly the same: where and are the widths of the passband and stopband regions, respectively.

##### 2.4. Design Method 4 (DM4)

The fourth method is to find the optimal matrix filter which constrains the left stopband and the right stopband with and , respectively. When the interferences were differently on both sides, we could suppress them more precisely with different attenuation levels:

##### 2.5. Design Method 5 (DM5)

Consider

In this method, we derive the sum of the left stopband and the right stopband attenuations with the weighting factors 1 and , respectively, and the passband fidelity is restricted by less than . The weighting ratio of the right stopband and the left stopband attenuation was .

#### 3. Problem Solution

In this section, the mathematical solutions of computing the optimal spatial matrix filters are derived by using the Lagrange multiplier technique. A numerical technique based on the generalized singular value decomposition method is also proposed for reducing the computational complexity of determining the optimal Lagrange multipliers.

For design method 1, the Lagrangian for the problem is defined as where is the Lagrange multiplier. Because the cost function and the constraint are strictly convex, the Lagrangian is also convex in and is minimized for any by where , , denotes the conjugate transpose of a matrix, and denotes the inverse of a nonsingular matrix. Substitution of into (9) gives the dual function where denotes the trace of a matrix. The duality theorem states that the optimal Lagrange multiplier can be obtained by maximizing . It can be shown that the optimal spatial matrix filter and the equation of finding are as

It is interesting to note from (12) that as , . Hence, for a smaller value of , we obtain a larger Lagrange multiplier and vice versa. From (8), a larger value of implies that the stopband error weighted more heavily compared to the passband error. Hence, one would expect that a larger value of would lead to better stopband attenuation at the cost of increased passband ripple.

Similarly, for design method 2, the optimal spatial matrix filter and the equation of finding the optimal Lagrange multiplier are as follows:

For design method 3, the optimal spatial matrix filter with the general mean square error can be deduced by (11) with or by (13) with . The optimal spatial matrix filter is

Note that (13) can be reexpressed as

Comparing (11) with (16), it is clear from (11) and (13) that if , then the two optimal spatial matrix filters are identical. When the stopband attenuation has been specified with by (12) with a certain , the passband fidelity can be easily deduced by (14) with and vice versa. By using , (12), and (14), the relationship between the stopband attenuation and passband fidelity is

The optimal spatial matrix filter with general mean square error is obtained by design method 3. By substituting into (17), the general response error is

It can be seen that this filter is just the same as the filter which is proposed by MacInnes in [4]. In paper [4], the filter was given by the product of one matrix and the pseudoinverse of another matrix.

Similarly, the optimal solution of design methods 4 and 5 can be derived by using the Lagrange theory. The optimal solution and the equations for solving the optimal Lagrange multipliers and of design method 4 are given directly by the following:

The optimal solution of design method 5 and the equation for solving the optimal Lagrange multiplier are given as follows:

For a given division of the passband and the stopband, the correlation matrices , , , and could be obtained. If we get the optimal Lagrange multiplier, then we get the optimal spatial matrix filters by (11), (13), (19), and (22). In other words, the spatial matrix filter design problems had been changed into the problems of solving the nonlinear equations of (12), (14), (20), (21), and (23). There were only 1 (DM1, DM2, and DM5) or 2 (DM4) parameters that need to be solved. The optimizations of the previous methods besides [4] (the optimal filter was the same as that of DM3) were very complicated. The unknown filter has unknown parameters that need to be solved. In addition, with the increasing of the discrete points in the passband and the stopband region, the computational complexity of other methods increased greatly, whereas it had little effect on the proposed methods. The proposed methods were very efficient for spatial matrix filter design.

Noticing that all the equations for solving the multipliers were monotonous nonlinear functions, for the given stopband attenuation level and the passband fidelity level, the Lagrange multipliers were unique. They could be obtained by the most iterative root finding algorithm, whereas the other methods need some professional optimization software. In this paper, the method of dichotomy was used.

#### 4. Improving the Design Efficiency of Design Methods 1 and 2

The efficiency of designing the optimal matrix filter in design method 1 or in design method 2 is mainly influenced by determining the optimal Lagrange multiplier or . The computational complexity can be reduced significantly by using the generalized singular value decompensation method [13–15]. Since and are nonsingular Vandermonde matrices, there exist unitary matrices and and a nonsingular matrix such that where , , and ,.

By substitution of (24) into (11), (13), and (15), the optimal spatial matrix filters can be given by

For design methods 1 and 2, the optimal Lagrange multipliers and are the roots of the following equations, respectively:

Any root finding method can be used to solve for and which satisfy the equations defined by (26) and (27), respectively. Note that if (12) or (14) is solved directly using some iterative methods, it requires a matrix inversion or for each iteration. It is of the order of computations. On the other hand, using (26) or (27), the computation load becomes trivial because the matrix inversion involves a diagonal matrix. The generalized singular value decomposition of two matrices is the only price needed to be paid for the computation load. However, this needs to be done only once. Therefore, the computation efficiency can be greatly enhanced.

#### 5. Computer Simulation

In this section, the passband and stopband of all the filters are specified with and . The filters have the same dimension with .

Figure 1 showed the filters design by using DM1 and DM3. The stopband fidelities of DM1 filters were 0.032, 0.01, 0.001, and 0.00032, respectively, which were , , , and in decibels. The passband fidelity and stopband attenuation level of the filter designed by DM3 were 0.0222 and 0.0026, respectively, which were and in decibels. As we know, filter of DM3 has the general mean square response error, and it was obvious that the filter of DM3 can be obtained by using DM1 with properly set stopband attenuation level.

**(a) Power response**

**(b) Mean square error**

In paper by Zhu et al. [8], a semi-infinite optimization programming was used to obtain the optimal matrix filter. Although the filter was designed for digital filtering, it was used directly for spatial filtering in direction of arrival estimation. Yan and Ma [3] designed the specific spatial matrix filter for array signal preprocessing in direction of arrival estimation (Figure 2). Figure 3 compared the two filters designed above. As we could see there was a negligible difference between the two filters. And we need only to compare the Zhu filter with the proposed filter to illustrate the effectiveness of the proposed method.

**(a) Power response**

**(b) Mean square error**

**(a) Power response**

**(b) Mean square error**

Figure 3 showed the filters designed with Zhu method [8], DM1, and DM2. The filters by Zhu method and DM1 had the same average attenuation level with in the stopband. In DM2, the passband fidelity was , which was the same as that of the filter obtained by using Zhu method. It could be seen from Figure 3, on the one hand, that the filter of DM1 has much less passband response error than that of Zhu method. On the other hand, the filter of DM2 has much lower stopband attenuation than that of Zhu method.

As we had discussed above, by a smaller value of , we obtain a larger Lagrange multiplier and vice versa. Similarly, by a smaller value of , we could also obtain a larger Lagrange multiplier and vice versa. Figure 4 gave the relationship between the optimal Lagrange multiplier and the passband fidelity or the stopband attenuation, in which the filters were designed by using DM1 and DM2. It was obvious that the relation models between them were nonlinear monotonous functions. The optimal Lagrange multiplier was uniquely for the given passband fidelity or stopband attenuation. The multipliers could be obtained by the most iterative root finding algorithm. The dichotomy method was used here for solving the equations.

Figure 5 gave the characteristics of the spatial matrix filters designed by DM4. The left stopband attenuation level was 0.01, in decibels. The right stopband attenuation levels were 0.032, 0.01, and 0.0032, respectively, which were , , and in decibels, respectively. Compared with the first three methods, design method 4 could restrain the left stopband and the right stopband separately, while the methods of design methods 1 to 3 could only treat the stopband as a whole. The filters of DM4 were more effective when the interferences we need to suppress had different signal-to-noise ratio in the left and right stopbands.

**(a) Power response**

**(b) Mean square error**

The relationships between the optimal Lagrange multipliers , , , , and and the stopband attenuation or the passband fidelity , , , , and correspondingly were monotonic nonlinear. Figure 6 gave the relationship between the optimal Lagrange multiplier and the passband fidelity , in which the spatial matrix filters were designed by DM5. The right stopband attenuation weighting factors were ,,,, and . It could be seen from Figure 6 that one could obtain better passband fidelity by increasing the Lagrange multiplier.

#### 6. Conclusion

An efficient technique of designing spatial matrix filter based on convex programming for array signal preprocessing was proposed. Five methods were considered. The mathematical solutions of computing the optimal spatial matrix filters were derived by using the Lagrange multiplier technique. A numerical technique based on the generalized singular value decomposition method was also proposed for reducing the computational complexity of determining the optimal Lagrange multipliers of the first two design methods. By simulation, it could be found that the proposed technique was effective for designing spatial matrix filter.

#### Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.