Abstract
An efficient neural network-based approach for tracking of variable number of moving electromagnetic (EM) sources in far-field is proposed in the paper. Electromagnetic sources considered here are of stochastic radiation nature, mutually uncorrelated, and at arbitrary angular distance. The neural network model is based on combination of probabilistic neural network (PNN) and the Multilayer Perceptron (MLP) networks and it performs real-time calculations in two stages, determining at first the number of moving sources present in an observed space sector in specific moments in time and then calculating their angular positions in azimuth plane. Once successfully trained, the neural network model is capable of performing an accurate and efficient direction of arrival (DoA) estimation within the training boundaries which is illustrated on the appropriate example.
1. Introduction
Signal source localization by employing passive antenna arrays is widely used technique in different areas such as communications, radars, acoustics, and medicine. Important step in this spatial determination of source location is to perform an angular direction of arrival (DoA) estimation of a signal radiated from the source. Among other things, the purpose and nature of the signal have to be taken into account while performing the DoA estimation, as signals can be considered either desired and deterministic or interfering both deterministic (unintentional interference) and stochastic (random function in time). In wireless communications, once the angular positions of desired/interfering electromagnetic (EM) source are found by using DoA estimation, the adaptive beam-forming algorithm can be employed to optimize the radiation pattern of antenna array so that it allocates the main beam towards the user of interest and generates deep nulls in the directions of interfering signals from mobile users in adjacent cells.
A number of DoA estimation algorithms have been proposed in the literature taking into account the statistical properties of source signals, geometry of the antenna arrays at the receiver end, multiplexing schemes, and so forth. Majority of these algorithms rely on the processing of a spatial covariance matrix of received signals at antenna array elements. Multiple Signal Classification (MUSIC) [1] is one of these techniques, widely used due to its superresolution capabilities. However, it is of high computational complexity as it requires a demanding spectrum search procedure, resulting in some cases in a longer run time not suitable for real-time applications. Artificial neural networks (ANNs) [2–4] represent an alternative faster approach to the MUSIC and other intensive superresolution DoA algorithms. ANNs are very convenient as a modeling tool since they have the ability to learn from the presented data and therefore they are especially useful in solving complex problems or those not fully mathematically described. In other words, ANNs are able to map dependence between two datasets. The learning process is an optimization procedure through which parameters of the ANN are optimized to have the ANN outputs as close as possible to the target values. This ability qualifies ANNs as very suitable tool for estimating the angular positions of source signals [4, 5].
In [6] a new approach based on combination of the Multilayer Perceptron (MLP) [3, 4] and the Radial Basis Function (RBF) ANNs [3, 4] is developed for two-dimensional, in azimuth and elevation planes, DoA estimation of deterministic signals radiated from narrowband EM sources. In [7, 8] and in [9], which was extended version of [8], an ANN approach, realized by the MLP neural model, has been presented to provide a high-resolution DoA estimation of stochastic signals. Since no amplitudes can be defined for the numerical values of stochastic signals, the characterization of stochastic signals differs from the characterization of deterministic signals. It requires considering the correlation between any two spatial points of the stochastic source in order to provide an estimation of spatial covariance matrix. A network-based methodology for the numerical computation of stochastic electromagnetic (EM) fields excited by spatially distributed noise sources with arbitrary spatial correlation was presented in [10, 11]. Based on stochastic source radiation model developed from [10], the MPL models from [7–9] were able to efficiently perform mapping from the space of stochastic signals described by the correlation matrix to the space of DoA in angular azimuth coordinates. However, their application was limited to the cases of only few stochastic narrowband EM sources in the far-field, at the fixed mutual distance. In [12–14], the developed MLP models were extended to allow an efficient DoA estimation of a number of mutually arbitrary positioned uncorrelated stochastic EM sources in far-field.
Both the superresolution algorithms and previously mentioned neural models have one limitation when performing the DoA estimation. The number of EM sources presented in the observed sector has to be known in advance in order to preserve model validity and its sufficient accuracy for angular positions determination. If the model is developed for particular number of sources assumed to be present in the observed sector during the model operation, in cases when the actual number of present sources is smaller or higher than assumed number, it is possible that model will incorrectly identify sources angular positions. Therefore in this paper, two-stage neural model, based on combining the probabilistic neural network (PNN) [15, 16] and the MLP network, is proposed in order to overcome this limitation. The PNN-MLP model is capable of performing an efficient and accurate DoA estimation of stochastic EM sources whose number is changing in time and sources are also moving fast in the observed sector. The example presented in the paper demonstrates the accuracy and suitability of the proposed neural network model for real-time applications.
2. Stochastic Source Radiation Model
Stochastic source radiation model, presented in [7–9, 12–14] and also used in this paper, starts from the assumption that each source radiation in far-field can be represented by linear uniform antenna array with elements mutually separated by , , where is observed frequency in far-field (Figure 1). In general, the degree of correlation between antenna elements feed currents, described by vector , is arbitrary and it can be expressed by the correlation matrix [10, 11]:By employing the Green function marked with vector , where and are azimuth and elevation angles determined with respect to the first antenna element, the level of electric field radiated from the antenna array representation of stochastic source, at some sampling point in the far-field, can be calculated as is the radiation pattern of antenna array, is the distance of far-field point to the centre of array, is free-space impedance, is the phase constant (), and are the distances of considered far-field point from the first to the th element of antenna array. For observation points in the far-field, we use a more general notation in order to describe the antenna array elements distance from particular points in far-field. For example, in Figure 1 represents the distance between th element () in the antenna array and th point in the far-field ().
For sampling points () in far-field scan area, determined by the azimuth and elevation plane angles , the correlation matrix of signals received in these sampling points can be obtained as [10]
For more than one stochastic source, the EM field level in far-field sampling point, as well as the elements of correlation matrix, can be determined by the superposition of radiation from all sources. If the number of stochastic sources is , then the vector has a formwhere is the distance between th element in antenna array, representing th stochastic source, and the sampling point in far-field, while the feed currents vector iswhere is the feed current of th element in antenna array representing th stochastic source. Incorporating (4) and (5) into (3) it is possible to determine the elements of correlation matrix . When the degree of correlation between antenna elements feed currents is unknown, its correlation matrix can be obtained by near-field measurements as described in [10, 11]. If two radiation sources that are under monitoring have the same angular position , but at the different distances and , and are represented with antenna arrays with , elements for the first and the second source, respectively, in case , then . By normalization of elements of matrix with respect to the first element , the matrix is obtained and its elements do not depend on the values of and . During the neural model development, only the first row of spatial correlation matrix has to be used, because it was shown that it contains sufficient information to be extracted by the neural model in order to estimate the source angular position [6, 7].
3. PNN-MLP Model
The main purpose of the PNN-MLP model presented in this paper is to determine in real time, based on sampled values of spatial correlation matrix , angular azimuth positions of stochastic EM sources, which can move fast in the observed spatial sector and also their number can vary in time. The architecture of this model is chosen so that calculations are performed in two stages: at the first stage (PNN stage) the number of stochastic EM sources that are currently present in the observed sector is determined, while at the second stage (MLP-DoA stage), based on information obtained from the first stage, the angular azimuth positions of sources in the sector are estimated (Figure 2).
The PNN stage consists of PNN with one output that gives the estimated number of EM sources in the observed sector , , where is the maximal number of sources whose positions can be simultaneously determined in the azimuth plane. PNN intended for such classification will be marked in further text as PNN-SND (PNN for Source Number Determination). The second, so-called MLP-DoA stage, consists of a bank of MLP networks intended for DoA estimation of EM sources and a switch for selection (activation) of appropriate MLP network from the bank. Bank of MLP networks contains networks: , where network has in total outputs and as such it performs calculation of angular positions of sources in the case when in the observed sector and in considered moment in time there are precisely stochastic sources. Switch has a task to select in chosen moment in time, for obtained number of sources , an appropriate MLP network for DoA estimation, that is, .
The training of PNN-MLP neural model is conducted in such way so that PNN-SDN network and each network are trained independently by using their own training set. The same applies for the testing phase; however it is useful to perform a testing of the integral PNN-MLP model as well with the goal to evaluate the model performances in real operating mode.
3.1. PNN for Source Number Determination (PNN-SND)
The architecture of neural network for determination of the number of stochastic EM sources in the observed spatial sector (PNN-SND) is based on PNN [15, 16] and it is shown in Figure 3. It consists of one input layer, one hidden layer, one class layer, and one output layer, that is, decision layer. The task of this neural network is to perform the classification of samples of the first row of spatial correlation matrix, that is, to determine which class among the predefined classes of this problem the sample at the network input belongs to. Regarding this problem, the classes are predefined in the following way:(1)There are in total classes of samples where is the maximal number of stochastic EM sources that can be found in the observed sector.(2)A sample of the first row of matrix belongs to the class where when it is sampled for the case when there were stochastic sources present in the sector.
According to this, PNN-SND performs mapping of the sampled values of the first row of the matrix into the set of discrete values of notations of predefined classes
The input layer of PNN-SND is the buffer layer, and it has the task to forward the values of the first row of the correlation matrix to each neuron in the hidden layer. For each element from the matrix row there are two neurons from the input layer of neural network that correspond, one for real and the other for imaginary part of element:so that the vector of dimension , which represents the vector of buffered values of network input, is forwarded to the input of each neuron in the hidden layer.
The hidden layer of PNN-SND is carrier information about the classes. Neurons in the hidden layer are divided into group of neurons so that each class , where , has its own group of neurons. The number of neurons within each group , , is determined during the training phase of neural network and it is equal to the number of samples in the training set that belongs to class . Activation function of neuron in the hidden layer that belongs to the class is based on the Gaussian function [15, 17] so that the outputs of neurons of class are given aswhere vector of dimension represents the vector of weights or vector of center of activation function of th neuron that belongs to class , while is the spread parameter (standard deviations) of activation function.
The task of neurons in the class layer is to sum the outputs of neurons in the hidden layer within each class separately and based on this summation to estimate the probability that sample belongs to considered class. Each group of neurons of class , , from the hidden layer corresponds to one neuron in the class layer (th neuron) so that the total number of neurons in the class layer is equal to . The estimation of probability that the sample belongs to class , , is performed by the th neuron in the class layer through its activation function based on Parzen window technique [17, 18] so that the outputs of this layer are given as
Output layer or decision layer has one neuron that has to decide, based on estimated probabilities in the class layer, to which class the sample at the network input is the closest, that is, where this sample has to be correctly classified. This neuron performs this task according to Bayes’s decision rule [19] based on the output of all the class layer neurons so that the activation function has a competitive nature; that is, as a final decision, the class for which the estimated probability is the highest is selected:
During the training of PNN-SND network, the numbers of neurons in the hidden and class layers are determined based on the training set, and also the weighting vectors in the hidden layer are adjusted so that the neural network performs correct classification of all samples from the training set. Spread parameter is not determined during the network training as its value is set before the training. The value of this parameter has an impact on generalization capabilities of PNN-SND and this impact can be quantified through the number of incorrectly classified samples by the network during the testing phase on the set of samples not used for the training. By multiplying repetition of network training for different values of spread parameter (typically in the range ) and by evaluating the network performances during the testing phase, the value of spread parameter can be adjusted so that the network during the testing has as smaller as possible the number of incorrectly classified samples.
3.2. MLP-DoA Network
The main task of network is to perform the mapping from the space of signals described by first row of the correlation matrix to the space of DoA in azimuth; that is,where is azimuth angles vector of stochastic sources . In the observed case elevation coordinates of radiation sources are neglected. The architecture of developed neural model is shown in Figure 4.
Its MLP-DoA network can be described by the following function:where vector represents the output of th hidden layer, is a connection weight matrix among th and th hidden layer neurons, and is a vector containing biases of th hidden layer neurons. is the activation function of neurons in hidden layers and in this case it is a hyperbolic tangent sigmoid transfer function:
Following the previously used notation, represents the input layer of MLP-DoA network so that . Also, is given as , where is a connection weight matrix between neurons of last hidden layer and neurons in output layer. The optimization of weight matrices and biases values during the training allows ANN to approximate the mapping with the desired accuracy. General notation for architecture of MLP-DoA network is , where represents the total number of the hidden layers, while represents the number of neurons in th hidden layer.
4. Modeling Results
In order to verify the proposed approach, the PNN-MLP neural model is applied for determination of azimuth positions of variable number of stochastic EM sources that arbitrary changes their positions. The model is realized within the MATLAB software environment. The observed sector is defined within the limits and in this sector at any time, the maximal number of radiation sources is assumed to be three (). According to this, for construction of PNN-MLP model, PNN-SND network is first used in order to classify samples within the three classes (). In addition, MLP-DoA1, MLP-DoA2, and MLP-DoA3 networks are used to determine the angular positions of one, two, and three sources, respectively, in azimuth plane. The training and testing of these networks are conducted independently; each network has its own training and testing sets. The common conditions under which the training and testing sets are generated are given in Table 1.
For generating the training and testing sets for all models, (3) and (4) are used as they perform the inverse mapping from the mapping done by the PNN-MLP modeland then, because the neural networks do not support operation with complex numbers, the first row of the correlation matrix is, according to (6) and (11), reconverted into the input vector of PNN and MLP networks, whose values are then used during the training
For this mapping, it was assumed that the all feed currents of antenna array elements, representing the stochastic sources, are mutually uncorrelated, so that is the unit diagonal matrix. In order to generate the training and testing sets, for each element of vector uniform distribution of samples for azimuth angles of radiation source location of the form is used, where and represent the lowest and highest limit of distribution, while is uniform sampling step. The sampling step is used for all the training and testing sets where, by adjusting the parameter , the size of sampling set can be determined as well as the level of overlapping between the training and testing sets. The main criteria for choosing the value of parameter for training and testing sets generation were to minimize the overlapping between these two sets in order to obtain the real estimation of the achieved level of generalization of trained network. During the generation of training and testing sets, in cases when there are more than one source in the observed sector, the samples, where angular positions of two and more sources are overlapped, are removed from the sets. Having in mind the training procedure and accuracy of neural networks it is best to treat these sources of the same angular positions as a unique source and this special case is considered as a case of smaller number of sources whose angular azimuth positions are not overlapped.
4.1. Training and Testing of PNN-SND
For training and testing of PNN-SND network, the sets of forms are used, where represents the number of present sources in the sector during the generation of values . The sampling sets for training and testing are obtained as
The set for network training is generated for values , , and so that the training set of 8921 samples is obtained. Testing set is generated for values , , and giving in total 7556 test samples. The training of PNN-SDN is conducted for different values of spread parameter in the range with 0.05 steps. Criteria for the best trained networks were the percentage of incorrectly classified samples. The testing results for the four best trained networks are shown in Table 2. Notation for trained PNN-SDN is PNN-spread where instead of word spread the value of spread parameter, used during the network training, is given. Based on these results (Table 2), the PNN-0.10 network is used to realize the PNN-MLP model.
From Table 2, it can be seen that some networks have relatively low percentage of incorrectly classified samples (below 4%) which illustrates potentially high performances of PNN and justifies their selection for realization of PNN-MLP model.
4.2. Training and Testing of MLP-DoAs Networks
For training and testing of all three types of MLP-DoA networks, , , the sets of forms are used, where represents the number of sources in the observed sector for which the MLP-DoA network performs DoA estimation. During the training phase, MLP networks with two hidden layers are used. In order to obtain the training network of highest possible accuracy, for each type of MLP-DoA network, the training of higher number of MLP-DoA networks with different number of neurons in the hidden layers is conducted. The training of all MLP-DoA networks is performed by using the quasi-Newton training method with given accuracy of 10−4. For the selection of best trained networks, the following statistical parameters are considered during the testing phase: maximal error during the testing phase (Worst Case Error, WCE), the average testing error (ATE), and Pearson Product-Moment correlation coefficient [3, 4].
4.2.1. Case
The sets for training and testing of MLP-DoA1 network are obtained asThe set for network training is generated for giving in total 121 samples. Testing set is generated for and therefore it has 69 samples.
The testing results for six trained MLP-DoA1 networks with the best test statistics are shown in Table 3. MLP2-23-23 network which shows the lowest Worst Case Error is chosen as representative MLP-DoA1 neural network. The scattering diagram of MLP2-23-23 network output and output of the MUSIC model obtained by model simulation on the same testing set is shown in Figure 5. A good agreement can be observed between the output values of neural network and referent azimuth values. It can be noticed that the MUSIC model has slightly better agreement with referent values which is expected; however it requires significantly longer run time than neural network (which can be seen later in the paper in Table 6).
4.2.2. Case
The sets for training and testing of MLP-DoA2 network are obtained asIn order to generate the training and testing sets, and are used, respectively. Therefore, the training set has 1830 samples, while the number of testing samples is 1176.
The testing results for six trained MLP-DoA2 networks with the best test statistics are shown in Table 4. MLP2-13-13 network which shows the lowest Worst Case Error is chosen as representative MLP-DoA2 neural network. The scattering diagram of MLP2-13-13 network outputs and outputs of MUSIC model obtained by model simulation on the same testing set are shown in Figures 6 and 7. A very good agreement between the output values of neural network and referent azimuth values can be observed. As in the previous case, the MUSIC model has a better agreement with the referent values; however, it requires much longer run time (which can be seen later in the paper in Table 6).
4.2.3. Case
The sets for training and testing of MLP-DoA3 network are obtained asThe training and testing sets are generated for and , respectively, giving in total 4495 training and 816 testing samples.
The testing results for six MLP-DoA3 trained networks with the best test statistics are shown in Table 5. MLP2-22-22 network which, among the group of models with the highest values, shows the lowest Worst Case Error is chosen as representative MLP-DoA3 neural network. The scattering diagram of MLP2-22-22 network outputs and outputs of MUSIC model obtained by model simulation on the same testing set are shown in Figures 8, 9, and 10. Again, the MUSIC model produces the best agreement with the referent values. Neural model has slightly worse agreement in comparison to the MUSIC model; however, its accuracy is still very good and it is achieved with significantly shorter run time than the MUSIC model.
4.3. Comparison between MLP-DoAs and MUSIC Models Run Times
As an illustration of networks efficiency for DoA estimation, comparison of simulation run times required for these networks to calculate the angular positions of radiating sources in points defined by the testing set with the MUSIC model simulation run time is shown in Table 6. In this comparison, the MUSIC model with space scan resolution of 0.1° is used. The run times given in Table 6 are rounded and they are measured for simulations running on referent hardware platform Intel Pentium M processor 1.73 GHz, 512 MB RAM, within the MATLAB software package.
It can be seen that the neural network performs DoA estimation at much higher speed than the MUSIC model. This neural network ability is very important for choosing an appropriate model for fast real-time DoA applications.
4.4. PNN-MLP Model Simulation of DoA Estimation of Three Mobile Sources
After the training and selection of PNN and appropriate networks according to the testing results, the PNN-MLP model is realized. Within the MATLAB software environment, the simulation of movement in real time of three independent stochastic sources in the observed sector is done and in real time determination of angular azimuth positions of these sources is performed by using the realized PNN-MLP model. The first, second, and third sources are moved on the trajectories , , and , respectively, where represents the time interval in seconds passed after the time when PNN-MLP model has started a sector monitoring. Sector monitoring and DoA estimation performed by this neural model lasted 100 s. The movement of sources was selected in such way so that during the monitoring there were time intervals of one, two, or three sources present in the observed sector. The results of DoA estimation are shown in Figure 11. Great reliability of the neural model to determine the number of sources in the observed sector and in different moments in time can be observed. In addition to that, the determination of angular positions in azimuth plane of sources present in the observed sector in different moments in time is achieved with a high accuracy.
5. Conclusions
The neural network-based approach for DoA estimation of EM radiation of variable number of moving stochastic sources is presented in the paper. Two different neural networks, PNN and MLP, are used to create the neural model capable of accurately and efficiently determining the angular positions of sources in case when its number is changing in time due to their movement in the observed sector. Considered example verifies that proposed neural model avoids intensive and time-consuming numerical calculations in comparison to the conventional approaches and therefore it is more suitable than conventional approaches for real-time applications.
By analysing the results presented in the paper, a potential problem while using the PNN-MLP model could appear. Increase in maximal number of sources for which the DoA estimation can be performed might lead to the higher complexity of PNN network and the greater total number of MLP-DoA networks in the neural model. Therefore, the overall architecture of the neural model becomes more complex making it difficult to train and reducing its accuracy for the case of greater number of radiating sources in the observed sector. Future research will be focused to solve this potential problem.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This research work has been supported by the Ministry for Education, Science and Technological Development of Serbia. Also, it has been done within the framework of COST Action IC1407 (COST ACCREDIT).