Abstract
Nearfield wideband beamformers for microphone arrays have wide applications in multichannel speech enhancement. The nearfield wideband beamformer design based on convex optimization is one of the typical representatives of robust approaches. However, in this approach, the coefficient of convex optimization is a constant, which has not used all the freedom provided by the weighting coefficient efficiently. Therefore, it is still necessary to further improve the performance. To solve this problem, we developed a robust nearfield wideband beamformer design approach based on adaptive-weighted convex optimization. The proposed approach defines an adaptive-weighted function by the adaptive array signal processing theory and adjusts its value flexibly, which has improved the beamforming performance. During each process of the adaptive updating of the weighting function, the convex optimization problem can be formulated as a SOCP (Second-Order Cone Program) problem, which could be solved efficiently using the well-established interior-point methods. This method is suitable for the case where the sound source is in the nearfield range, can work well in the presence of microphone mismatches, and is applicable to arbitrary array geometries. Several design examples are presented to verify the effectiveness of the proposed approach and the correctness of the theoretical analysis.
1. Introduction
As one of the main applications of the microphone array, beamforming techniques have been widely used in speech input devices in computers, hearing aids, hands-free telephony, teleconferencing, robot hearing, biomedical engineering, and so forth [1]. In most of these applications, the target source is located in the nearfield range. In this case, the conventional approaches based on farfield assumption to design the beamformers will result in significant degradation in the beamforming performance [2–4]. For this reason, nearfield beamforming based on microphone array has attracted a growing number of researches. So far, many nearfield beamforming approaches have been proposed [5]; however, most of them only consider the ideal microphone array characteristics without errors. Actually, the beamformers are highly sensitive to the errors in microphone array characteristics, that is, microphone gain, phase, and position errors. Therefore, the efficient and robust design of the nearfield wideband beamformers for microphone arrays is practically required [6, 7]. An existing typical representative of robust designs is nearfield wideband beamforming based on convex optimization [8–10]. To some extent, the existent approach has suppressed the degradation caused by errors in microphone array characteristics. However, in this approach, the coefficient of convex optimization is a constant, which has not used all the freedom provided by the weighting coefficient efficiently. That is the reason why it is still necessary to improve the performance further.
To solve the above problems, we developed an approach for the design of nearfield wideband beamformers robust against errors in microphone array characteristics in this paper. Our robust nearfield beamformers are designed based on adaptive-weighted convex optimization. The proposed approach defines an adaptive-weighted function by the adaptive array signal processing theory and adjusts its value flexibly, which has improved the beamforming performance. And it can work well in the presence of microphone mismatches and is applicable to arbitrary array geometries.
2. The Conventional Nonrobust Design: Minimax Nearfield Wideband Beamformers
In order to facilitate the analyses below, this section introduces a conventional minimax beamformer design.
Without loss of generality, let us consider a uniform linear microphone array with elements receiving a wideband signal from a nearfield location. Set as the th element’s position vector, and the configuration of the filter-sum beamformer is shown in Figure 1. Let an -tap FIR filter be used behind each microphone element. Filter the -way signals and sum them together, and then the output signal is obtained.
The actual array response to the source signal from location at frequency can be expressed aswhere is a vector of FIR filter weights, that is, a beamformer weight vector, the superscript represents the transpose, is an array response vector, andwhere stands for Kronecker product andwhere is a transfer function from the source point to the th microphone at frequency and is a sampling frequency. With the ideal microphone characteristics, is given by [7]where is the sound speed in the air and denotes the Euclidian norm.
Given the arbitrary desires response , the minimax design approach for nearfield wideband beamformers can be formulated as [11]where is a positive weighting coefficient and is a predefined space-frequency domain of interest. We assume that is a convex set hereafter. Problem (5) can be formulated as the following semi-indefinite convex programming; that is,
Problem (6) may be called a SOCP problem and thus be solved efficiently using the well-established interior-point methods [12].
3. Existing Robust Design: Nearfield Wideband Beamformers Based on Convex Optimization
To compare with the proposed approach, in this section, the existing robust approach is introduced.
In reality, due to temperature, humidity, vibration and working time, the production process, and other factors, the characteristics of the microphone usually change over time; in other words, there are mismatches in the microphone array. As a result, these will lead to the distortion of the array response vector . We assume that the norm of the distortion can be bounded by some known positive constant ; that is,where is a constant as small as possible.
Hereinafter, we will derive the lowest bounds on the parameter with considering the uncertainty of microphone array characteristics.
The minimax design of the existing robust approach can be formulated as [13]
Formula (8) can be calculated as follows:
When , the equality in (9) holds. So,
According to (10), the optimization problem in (8) can be expressed as
Problem (11) can be transformed into a convex optimization problem; that is,
Problem (12) may be called a SOCP problem and thus be solved efficiently using the well-established interior-point methods.
4. The Proposed Robust Design: Nearfield Wideband Beamformers Based on Adaptive-Weighted Convex Optimization
Compared with the conventional minimax approach, the existing approach has considered the mismatch of the microphone array, which improves the beamforming performance to some extent. However, since the coefficient of the convex optimization is a fixed constant, the design has to be improved further. In this section, a robust design approach based on adaptive-weighted convex optimization is proposed. During the convex optimization, our approach adopts an adaptive-weighted function or interactively optimizing method instead of the fixed constant, which can change flexibly according to the design needs. This is because the adaptive or interactive optimization method can adaptively change with the microphone array environment factors, such as temperature, humidity, vibration and working time, the production process, and other factors, of uncertainty, so it can effectively overcome the lack of flexibility and adaptability of the constant weighting coefficient in convex optimization and improve the weighted coefficient of freedom with strong selectivity and adaptability. Accordingly, it is beneficial to improving the performance of the array response.
4.1. Design Guidelines
The minimax design of the proposed robust approach can be formulated aswhere is the weighting function that can change according to the location at frequency , and its adaptive update process will be elaborated hereinafter in detail.
Using (7) to simplify problem (13), problem (13) can be formulated as follows:
To simplify the calculation, let us transform problem (14) into a convex optimization problem; that is,
Problem (15) can be formulated as a SOCP problem and thus be solved efficiently using the well-established interior-point methods. The process will be elaborated hereinafter in detail.
4.2. Lowest Bound on
In this section, we will derive the lowest bounds on the parameter .
4.2.1. Lowest Bound on with Microphone Gain and Phase Errors
The characteristics of the th microphone with gain and phase errors can be represented aswhere and are the gain and phase errors of the th microphone, respectively. When the microphone gain and phase errors are considered, the transfer function in (4) becomes
So,
Suppose thatwhere and are the known bound.
It holds thatWhen , , the equation holds, and attains its maximal value. Therefore,
4.2.2. Lowest Bound on with Microphone Position Errors
Denote the position error of the th microphone as ; that is, the actual position of the th microphone is given bywhere we assume that .
When the microphone position errors are considered, the transfer function in (4) becomes
So,
Suppose that,where is the known bound and is the minimal wavelength in the frequency band of interest. It holds that
Notice thatWhen , the equation holds, and attains its maximal value. Therefore,
When , , the lowest bound on is given by
4.3. Adaptive Update Process of Based on Adaptive Array Signal Processing Theory
Based on adaptive array signal processing theory [13], the weighting function can be regarded as the interference signal strength incident on the array from location at frequency . It is known that the response of an adaptive array to an interference signal depends on the interference signal strength; moreover, the stronger the interference is, the lower the resulting beam pattern level is. Therefore, by controlling the weighting functions, it can be expected that the performance of the designed beamformer can be improved.
Let be the chosen grid that approximates the predefined space-frequency domain of interest . The adaptive update process for the weighting function in problem (15) is defined as follows: If else
where is the weight vector for beamformer during the th iteration, is the predefined permitted error between the designed array response and desired array response, and is an iteration gain.
Thus, for the th iteration, the optimization problem (15) can be formulated as
The weight vector is obtained via using the well-established interior-point methods in SeDuMi. The process will be elaborated hereinafter in detail.
To summarize, the proposed robust approach based on adaptive-weighted convex optimization consists of the following steps.(1)Discretize the specified space–frequency region .(2)Initialize the parameters , , , , and and the maximum iteration times .(3)Calculate using (2)~(3).(4)Calculate using (21) or (29).(5)For the th iteration,①calculate using (30)~(31);②using , (32) and interior-point methods update .
4.4. SOCP-Based Implementation
In this section, the process of solving the SOCP problem by the well-established interior-point methods in SeDuMi will be elaborated.
SeDuMi is a Matlab toolbox developed by Sturm to process symmetrical cone programming problems, which is very convenient to solve the convex optimization problems constrained by second-order and linear conditions. In order to achieve standardized and accurate high effect, we use SeDuMi to deal with the beamforming optimization problem. In SeDuMi, the dual standard form of the SOCP is defined aswhere and are arbitrary real-valued vectors, is an arbitrary real-valued matrix, the vector contains the design variables to be solved, is the number of second-order cone (SOC) constraints, and the -dimensional SOC constraint is defined aswhereNext, we formulate our design problem (32) to the standard dual form of the SOCP (33), which can then be solved efficiently using SeDuMi. We define the following vectors:where ; let and hereafter denote the real and imaginary parts of a scalar/vector, respectively.
And we have the following matrices:where represents the identity matrix.
We can rewrite problem (32) as the dual standard form of the convex SOCP; that is,
After solving the optimization problem by SeDuMi, for the th iteration, the weight vector of the beamformer is the 3rd-()th components of vector .
5. Simulation Examples
Several design examples are presented to illustrate the performance of the proposed nearfield beamformers. If there are no special instructions, in this paper, the simulation conditions are as follows. A uniform linear array with 7 microphones is used. With interelement spacing of 5 cm, the microphones are located at , respectively. Behind each microphone, an FIR filter with taps is used. The sample frequency is set to , and the nominal value of the sound speed is . The desired response function in the passband is chosen as [8]
while in the stopband the desired response function is 0. In the following examples, the sound source is specified on an -axis parallel to, and 1 m in front of, the microphone array. The wideband signal to be processed has the frequency in the range . The passband region is defined as
And the stopband regions are defined aswhere is in meters and is in Hertz, respectively. The microphone gain errors have a uniform distribution in , the phase errors have a uniform distribution in , and the position errors have a uniform distribution in . When the mismatch in microphone array is considered, for each simulation run, the array response is evaluated, and each figure is the average of the array responses of 100 simulation runs.
5.1. Example 1: Minimax Nearfield Wideband Beamformer
In the first example, we consider the minimax design of nearfield wideband beamformer when there are no errors in microphone array characteristics. The array response of the minimax beamformer is shown in Figure 2. Here, we set . As is shown in Figure 2, under this ideal case, the array response has a good performance, which basically meets the design requirements. For comparison, Figures 3 and 4 show the array response of the minimax beamformer with microphone gain errors and position errors, respectively. Obviously, the beamformer deteriorates severely and even fails to work in the lower frequency band below 1500 Hz in Figure 3. The same situation also appears in Figure 4. Therefore, minimax beamformer fails to work in the case of considering microphone array mismatches.
5.2. Example 2: Existing Robust Nearfield Wideband Beamformer Based on Convex Optimization
5.2.1. Microphone Gain and Phase Errors
Set ; when considering the microphone gain and phase errors, the array response of the existing robust beamformer based on convex optimization is shown in Figure 5. Compared with the array response of the minimax beamformer shown in Figure 3, the existing approach has improved the performance of the beamformer, basically guaranteeing that the array response meets the design requirements over the entire band.
5.2.2. Microphone Position Errors
The array response of the existing robust beamformer is shown in Figure 6. Compared with Figure 4, the performance of the minimax beamformer is quite poor, while the existing robust beamformer produces a better result in the presence of microphone position errors.
5.3. Example 3: Proposed Robust Nearfield Wideband Beamformer Based on Adaptive-Weighted Convex Optimization
In this section, the performance of the proposed robust wideband beamformer will be considered. Set the iteration gain in the passband , the iteration gain in the stopband , the maximum iteration times , and the predefined permitted error . Initialize the parameters and as random numbers.
5.3.1. Microphone Gain and Phase Errors
Figure 7 shows the array response of the proposed robust approach when the microphone gain and phase errors are considered. Compared with Figure 3, Figure 7 shows that the proposed approach has significantly improved the performance of the beamformer, guaranteeing that the array response meets the design requirements over the entire band.
To gain a further comparison of the existing robust approach and the proposed one, we hereby introduce the concept of passband fluctuation error and stopband beam energy.
Firstly, we defined the passband fluctuation errors as the ratio of maximal magnitude response to the minimal magnitude response in the passband region. As is widely known, whether the beamwidth of different frequencies is the same is an important factor to measure the performance of a beamformer. Obviously, the smaller the passband fluctuation error is, the smaller the width of the beam in the passband is affected by the frequency and the less the beamwidth changes due to the different frequencies, and the stability of the array response is stronger. Figure 8 shows the curves of the passband fluctuation errors for both of the proposed approach and the existing approach (when the value for is different) with microphone gain and phase errors. Figure 8 shows that the passband fluctuation errors of the proposed approach are much less than the existing one. So, the proposed approach gives a better performance in the passband.
Now the performance of the two approaches in stopband will be evaluated. In the design of a beamformer, it is necessary to suppress the stopband, which means reducing the magnitude of the array response over the stopband region. In order to measure this factor, we defined stopband beam energy as the sum of energies of the array response in different positions at the same frequency. Obviously, the smaller stopband beam energy means stronger stopband suppression, which equals a better performance. Figure 9 shows the curves of stopband beam energy for both of the proposed approach and the existing one (when the value for is different) with microphone gain and phase errors. Figure 9 shows that the stopband beam energy of the proposed approach is much less than the existing one, which means the proposed approach gives a better performance in the stopband.
5.3.2. Microphone Position Errors
Figure 10 shows the array response of the proposed robust approach when the microphone position errors are considered. Compared with Figure 4, Figure 10 shows that the proposed approach has significantly improved the performance of the beamformer, guaranteeing that the array response meets the design requirements over the entire band.
Now, the comparison between the existing robust beamformer and the proposed robust beamformer is considered. Figure 11 shows the curves of the passband fluctuation errors of the proposed approach and the existing approach (when the value for is different) with microphone position errors. As can be seen from Figure 11, the passband fluctuation errors of the proposed approach are much less than the existing one, which means that the proposed approach gives a better performance in the passband.
Now the performance of the two approaches in stopband will be evaluated. Figure 12 shows the curves of stopband beam energy of the proposed approach and the existing one (when the value for is different) with microphone position errors. As can be seen from Figure 11, the stopband beam energy of the proposed approach is much less than the existing one, which means that the proposed approach gives a better performance in the stopband.
5.4. Example 4: The Influence of Array Configuration on Beamforming Performance
In this section, we consider the influence of different array configurations on the performance of the robust adaptive-weighted convex optimization broadband beamforming. Figure 13 shows the curves of the passband fluctuation errors with different array elements. From Figure 13, we can know that the fluctuation errors change from 24 dB to 28 dB with the increase of the number of the array elements, but this kind of change is not affected by frequency and has better stability within the beamwidth, and thus the stability of the array response is stronger. The longer the distance from the signal to the reference microphone is, the greater the fluctuation error is. Conversely, the greater the distance from the signal to the reference microphone is, the smaller the fluctuation error is; for example, when , m, the passband fluctuations are about 24 dB.
In the design of the beamformer, the stopband is suppressed and the response amplitude of the stopband is reduced as much as possible. The smaller the stopband beam energy is, the better the beamformer performance is. It can be seen from Figure 14 that the stopband beam energy curves have a little change when the number of array elements is different. Therefore, the performance of the proposed method is not affected by the number of array elements.
Figure 15 shows the curves of passband fluctuation errors of the proposed method with 7, 9, and 11 array elements, respectively. The passband fluctuation errors may increase with the increase of the number of the array elements and the distance from the source to the microphone, and thus the transfer function of the microphone can be affected, and the larger the number of elements is, the greater the passband fluctuation of errors is, but the smaller the passband fluctuation errors are.
Figure 16 shows the stopband beam energies with 7, 9, and 11 array elements of the proposed method, respectively. It can be seen from Figure 16 that the higher the element number is, the lower the stopband energy with the same stopband frequency is, and when the number of the array elements of the proposed method is as large as possible, the array response amplitude is reduced as much as possible.
6. Conclusions
In summary, compared with the existing robust nearfield wideband beamformer design approach, the proposed approach has the following advantages. When the mismatches in microphone array are considered, the proposed approach has better robustness and can meet the design requirements. To facilitate the analysis, the paper uses ULA example in the simulation. In fact, the proposed approach has no restriction on the structure of the array and can be applied to any microphone array geometry. Adaptive weighting coefficients in convex optimization, which makes full use of the freedom of design, can achieve better results.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (61371131, 61673222), the Major Project of Nature Science Foundation of Higher Education Institution of Jiangsu Province, China (Grant no. 13KJA510001), Jiangsu Province, and a project funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD II), as well as Top-Notch Academic Programs Project of Jiangsu Higher Education Institutions (TAPP, PPZY2015B134).