Quadratic Filtering Algorithm Based on Covariances Using Correlated Uncertain Observations Coming from Different Sensors
The least-squares quadratic estimation problem of signals from observations coming from multiple sensors is addressed when there is a nonzero probability that each observation does not contain the signal to be estimated. We assume that, at each sensor, the uncertainty about the signal being present or missing in the observation is modelled by correlated Bernoulli random variables, whose probabilities are not necessarily the same for all the sensors. A recursive algorithm is derived without requiring the knowledge of the signal state-space model but only the moments (up to the fourth-order ones) of the signal and observation noise, the uncertainty probabilities, and the correlation between the variables modelling the uncertainty. The estimators require the autocovariance and cross-covariance functions of the signal and their second-order powers in a semidegenerate kernel form. The recursive quadratic filtering algorithm is derived from a linear estimation algorithm for a suitably defined augmented system.
In many real systems the signal to be estimated can be randomly missing in the observations due, for example, to intermittent failures in the observation mechanism, fading phenomena in propagation channels, target tracking, accidental loss of some measurements, or data inaccessibility during certain times. Usually, these situations are characterized by including in the observation equation not only an additive noise, but also a multiplicative noise consisting of a sequence of Bernoulli random variables taking the value one if the observation is state plus noise, or the value zero if it is only noise (uncertain observations). Since these models are appropriate in many practical situations with random failures in the transmission, the estimation problem in systems with uncertain observations has been widely studied in the literature under different hypotheses and approaches (see e.g., [1, 2] and references therein).
On the other hand, in some practical situations the state-space model of the signal is not available and another type of information must be processed for the estimation. In the last years, the estimation problem from uncertain observations has been investigated using covariance information, and algorithms with a simpler structure than those obtained when the state-space model is known have been derived (see, e.g., ).
Recently, the least-squares linear estimation problem using uncertain observations transmitted by multiple sensors, whose statistical properties are assumed not to be the same, has been studied by several authors under different approaches and hypotheses on the processes (see, e.g., [4, 5] for a state-space approach and [6, 7] for a covariance approach).
In this paper, using covariance information, recursive algorithms for the least-squares quadratic filtering problem from correlated uncertain observations coming from multiple sensors with different uncertainty characteristics are proposed. This paper extends the results in  in two directions: on the one hand, correlation at times and between the random variables modelling the uncertainty in the observations is considered, and, on the other, the quadratic estimation problem is addressed. The quadratic estimation is also a new topic addressed in this paper over the results in Hermoso-Carazo et al.  which are also referred to observations with uncertainty modelled by Bernoulli variables correlated at times and with arbitrary , but coming from a single sensor. Furthermore, the current paper differs from  in the correlation model considered and in the information used to derive the algorithms (state-space model in  and covariance information in the current paper).
To address the quadratic estimation problem, augmented signal and observation vectors are introduced by assembling the original vectors with their second-order powers defined by the Kronecker product, thus obtaining a new augmented system and reducing the quadratic estimation problem in the original system to the linear estimation problem in the augmented system. By using an innovation approach, the linear estimator of the augmented signal based on the augmented observations is obtained, thus providing the required quadratic estimator.
The performance of the proposed filtering algorithms is illustrated by a numerical simulation example where the state of a first-order autoregressive model is estimated from uncertain observations coming from two sensors with different uncertainty characteristics correlated at times and , considering several values of . The linear and quadratic estimation error covariance matrices are compared, showing the superiority of the quadratic estimators over the linear ones.
2. Observation Model and Hypotheses
The problem at hand is to determine the least-squares (LS) quadratic estimator of an -dimensional discrete signal, , from noisy measurements coming from multiple sensors which may not contain the signal with different probabilities. In this section, we present the observation model and the hypotheses about the signal and noise processes involved.
Consider scalar sensors whose measurements at each sampling time, , denoted by , may either contain the signal to be estimate, , or be only noise, ; the uncertainty about the signal being present or missing in the observation is modelled by Bernoulli variables, . The observation model is thus described as follows: If , then and the measurement coming from the th sensor contains the signal; otherwise, if , then , which means that such measurement is only noise. Therefore, the variables model the uncertainty of the observations coming from the th sensor.
To simplify the notation, the observation equation (2.1) is rewritten in a compact form as follows: where , and .
It is known that if the signal and the observations have finite second-order moments, the LS linear filter of is the orthogonal projection of on the space of -dimensional random variables obtained as linear transformations of . So, by defining the random vectors ( denotes the Kronecker product ) and, if , the LS quadratic estimator of based on the observations up to the sampling time is the orthogonal projection of on the space of -dimensional linear transformations of and their second-order powers . To guarantee the existence of the second-order moments of the vectors , the pertinent assumptions about the processes in (2.1) are now stated.
2.1. Hypotheses about the Model(H1)The signal process has zero mean, and its autocovariance function, , as well as the autocovariance function of the second-order powers, , is expressed in a semidegenerate kernel form where the matrix functions , , and the matrix functions , , are known. Moreover, it is assumed that the covariance function of the signal and its second-order powers, , can also be expressed as where , , , and are , , , and known matrix functions, respectively. (H2)For , the sensor additive noises, , are zero mean white processes, and their moments, up to the fourth one, are known; we will denote , and . (H3)For , the noises are sequences of Bernoulli random variables with ; the variables and are independent for , and are assumed to be known. (H4)The signal process, , and the noise processes, and , where , are mutually independent.
3. Augmented System
Given the observation model (2.1) with assumptions (H1)–(H4), the problem is to find the LS quadratic filter of the signal , which will be denoted . The technique used to obtain this estimator consists of augmenting the signal and observation by assembling the original vectors and their second-order powers, , and deriving the estimator as the vector constituted by the first entries of the LS linear filter of based on .
To obtain this linear estimator, the first- and second-order statistical properties of the augmented vectors and are now analyzed.
3.1. Properties of the Augmented Vectors
By using the Kronecker product properties and denoting , and ( is the identity matrix and is the commutation matrix, ) the following model with uncertain observations is obtained, It should be noted that the signal, , and the noise, , in this new model have non-zero mean. Nevertheless, this handicap can be overcome by considering the centered augmented vectors and which, taking into account that , satisfy where being and the operator that vectorizes a matrix .
Note that the LS linear estimator of based on is obtained from the LS linear estimator of based on , just adding the mean vector . Hence, since the first components of are zero, the required quadratic estimator is just the vector constituted by the first entries of the LS linear filter of . Henceforth, these centered vectors will be referred to as the augmented signal and observation vectors, respectively.
The signal and noise processes and involved in model (3.3) are zero mean. In the following propositions the second-order statistical properties of these processes are established.
Proposition 3.1. If the signal process satisfies (H1), the autocovariance function of the augmented signal process can be expressed in a semidegenerate kernel form, namely, where
Proof. It is immediate from hypothesis (H1) about the covariance functions of the signal and its second-order moments.
Proposition 3.2. Under (H1)–(H4), the noise is a sequence of random vectors with covariance matrices, , given by where denotes the Hadamard product and with Moreover, is uncorrelated with the processes and .
Proof. It is obvious that . On the other hand, since , and , and are mutually independent, it is easy to see that and hence
Firstly, we prove that
where denotes the Kronecker delta function.
Indeed, since is a zero mean white sequence with covariances , it is clear that . Moreover, from the mutual independence, the Kronecker and Hadamard products properties lead to
On the other hand, and since for , the covariance matrices are obtained.
The uncorrelation between and the processes and is derived in a similar way, taking into account that , , and are mutually independent and using the Kronecker and Hadamard products properties.
4. Quadratic Filtering Algorithm
As indicated above, to obtain the LS quadratic estimators of the signal based on observations (2.1), we consider the LS linear estimators of the augmented signal, , based on the augmented observations (3.3). As known, the LS linear filter of is the orthogonal projection of the vector onto , the linear space spanned by ; so the Orthogonal Projection Lemma (OPL) states that the estimator, , is the only linear combination satisfying the orthogonality property
Due to the fact that the observations are generally nonorthogonal vectors, we will use an innovation approach, consisting of transforming the observation process into an equivalent process (innovation process) of orthogonal vectors , equivalent in the sense that each set spans the same linear subspace as ; that is, .
The innovation process is constructed by the Gram-Schmidt orthogonalization procedure, using an inductive reasoning. Starting with , the projection of the next observation, , onto is given by ; then, the vector is orthogonal to , and clearly . Let be the set of orthogonal vectors satisfying ; if now we have an additional observation , we project it onto and the orthogonality property allows us to find the projection by separately projecting onto each of the previous orthogonal vectors; that is, so the next vector, , is orthogonal to the previous ones and .
Note that the projection is the part of the observation that is determined by knowledge of ; thus the remainder vector can be regarded as the “new information” or the “innovation" provided by and the process as the innovation process associated with . The causal and causally invertible linear relation existing between the observation and innovation processes makes the innovation process unique.
Next, taking into account that the innovations constitute a white process, we derive a general expression for the LS linear estimator of the augmented signal, , based on , which will be denoted by . Replacing by the equivalent set of orthogonal vectors , the signal estimator is where the impulse-response function is calculated from the orthogonality property, which leads to the Wiener-Hopf equation Due to the whiteness of the innovation process, for and the Wiener-Hopf equation is expressed as consequently, and, therefore, the following general expression for the LS linear filter of the augmented signal is obtained where and .
Using the properties of the processes involved in (3.3), as established in Propositions 3.1 and 3.2, and expression (4.8) for the filter, we derive a recursive algorithm for the linear filtering estimators, , of the augmented signal . As indicated above, the first entries of these estimators provide the required quadratic filter of the original signal .
Theorem 4.1. The quadratic filter, , of the original signal is given by where is the operator which extracts the first entries of , the linear filter of the augmented signal , which is obtained by where the vectors are recursively calculated from The innovation, , satisfies with and , the covariance matrix of the innovation, verifies The matrix function is given by where is recursively obtained from
Proof. We start by obtaining an explicit formula for the innovations, , or, equivalently, for the one-stage predictor of , which by denoting is given by
Using the hypotheses on the model, it is deduced that
Using again the hypotheses on the model, we obtain
with given by expression (4.13).
Next, expression (4.10) for the filter is derived. For this purpose, taking into account expression (4.8), we obtain formulas to calculate the coefficients . From the hypotheses on the model, replacing by its expression in (4.21), and using (4.8) for , we have or, equivalently, where is a function satisfying Then, from (4.8) and (4.23), expression (4.10) for the filter is deduced, where , defined by satisfies the recursive relation (4.11). Analogously, it is obtained that the one-stage predictor of the signal is given by , which substituted into (4.21) leads to formula (4.12) for the innovation.
Expression (4.15) for is derived making in (4.24), using (4.23), and defining the function . From this definition, the recursive relation (4.16) is also immediately derived.
Finally, we obtain expression (4.14) for the innovation covariance matrix; from the hypotheses on the model, expression (4.12), and the definition of , the following equation is obtained: So, expression (4.14) for is deduced taking into account that , which follows from (4.11) using that the vector is orthogonal to .
To conclude, as a measure of the estimation accuracy, we have calculated the filtering error covariance matrices, , which clearly are obtained by .
5. Generalization to Correlation at Times and , with
The observation model considered in Section 2 assumes that the uncertainty is modeled by Bernoulli variables correlated at consecutive sampling times, but independent otherwise. In this section, such model is generalized by assuming correlation in the uncertainty at times and differing units of time. Specifically, hypothesis (H3) is replaced by the following one.(H3')For , the noises are sequences of Bernoulli random variables with . For , the variables and are assumed to be independent for , and are known for .
This correlation model allows us to consider certain situations where the signal cannot be missing at consecutive observations.
Similar considerations to those made in Section 3 for the case of consecutive sampling times lead now to the following expression for the covariance matrices of the noise : Then, performing the same steps as in the proof of Theorem 4.1, the following algorithm is deduced.
Theorem 5.1. The quadratic filter, , of the original signal is given by where is the operator which extracts the first entries of , the linear filter of the augmented signal , which is obtained by where the vectors are recursively calculated from The innovation, , satisfies with The covariance matrix of the innovation, , verifies The matrix function is given by where is recursively obtained from
Proof. It is analogous to that of Theorem 4.1, taking into account that, in this case, the one-stage predictor of satisfies and, from the model hypotheses, and for ,
6. Numerical Simulation Example
To illustrate the application of the proposed filtering algorithm a numerical simulation example is shown now. To check the effectiveness of the proposed quadratic filter, we ran a program in MATLAB which, at each iteration, simulates the signal and the observed values and provides the linear and quadratic filtering estimates, as well as the corresponding error covariance matrices.
For the simulations, this program has been applied to a scalar signal , generated by the following first-order autoregressive model: where the initial state is a zero mean Gaussian variable with and is a zero mean white Gaussian noise with .
The autocovariance functions of the signal and their second-order powers are given in a semidegenerate kernel form, specifically,
and the covariance function of the signal and their second-order powers is given by . According to hypothesis (H1), the functions constituting these covariance functions can be defined as follows:
Consider two sensors whose measurements, according to our theoretical study, are perturbed by sequences of Bernoulli random variables , , and by additive white noises, , ; that is,
Assume that the additive noises are independent and have the following probability distributions:
Now, in accordance with the proposed uncertain observation model, we assume that the uncertainty at any time is correlated with the uncertainty at time only if , but independent otherwise.
To model the uncertainty in this way, we can consider two independent sequences of independent Bernoulli random variables, , , and define . It is obvious that the variables take the value zero if and only if and ; otherwise, ; therefore, are Bernoulli variables with . Note that implies and, consequently, ; this fact implies that if the signal is missing at time , it is assured that, at time , the observation contains the signal; therefore, the signal cannot be missing in consecutive observations.
For the application, we have assumed that the variables in each sensor have the same distribution; that is, independent of . So, in each sensor, the probability that the observation contains the signal, , is constant for all the observations.
Since is independent of , also is independent of and hence, , for . For fixed , the variance of is , and the correlation between two different variables and is obtained as follows.(I)For the variables and are independent, thus being the variables and also independent and hence, uncorrelated; that is, .(II)For , since or , and . Then, , which implies that .
Summarizing, the correlation function of is given by
and, hence, the measurements above described are in accordance with the proposed correlation model.
To analyze the performance of the proposed estimators, the linear and quadratic filtering error variances have been calculated for different values of and also for different and , which provide different values of the probabilities and . Since are the same if the value is considered instead of , only the case is examined here (note that, in such case, is a decreasing function of ); more specifically, the values (which lead to , resp.) have been used.
First, considering , the linear and quadratic filtering error variances are calculated for the values , and , and . Figure 1 shows the results obtained; for all the values of , the error variances corresponding to the quadratic filter are always considerably less than those of the linear filter, thus confirming the superiority of the quadratic filter over the linear one in the estimation accuracy. Also, from this figure it is gathered that, as or increase (which means that the probability that the signal is present in the observations coming from the corresponding sensor decreases), the filtering error variances become greater and, hence, worse estimations are obtained.
Next, we compare the performance of the linear and quadratic filtering estimators for the values ; since the linear and quadratic filtering error variances show insignificant variation from the 5th iteration onwards only the error variances at a specific iteration are considered.
In Figure 2 the linear and quadratic filtering error variances at are displayed versus (for constant values of ), and, in Figure 3, these variances are shown versus (for constant values of ). From these figures it is gathered that, as or decrease (and, consequently, the probability that the signal is not present in the observations coming from the corresponding sensor, , decreases), the filtering error variances become smaller and, hence, better estimations are obtained. Note that this improvement is more significant for small values of or , that is, when the probability that the signal is present in the observations coming from one of the sensors is large. On the other hand, both figures show that, for all the values of and , the error variances corresponding to the quadratic filter are always considerably less than those of the linear filter, confirming again the superiority of the quadratic filter over the linear one.
Finally, for (these values produce the maximum value for the probability that the signal is not present in the observations coming from both sensors) and considering different values of , specifically , the error variances at for the linear and quadratic filters are displayed in Figure 4. From this figure it is deduced that the performance of the estimators improves when the values of are smaller and, hence, a greater distance between the correlated variables produces worse estimations (in the sense of the mean squared error). As expected, this figure also shows that the estimation accuracy of the quadratic filters is superior to that of the linear filters and also that the error variances show insignificant variation when the values of are greater.
A recursive quadratic filtering algorithm is proposed from correlated uncertain observations coming from multiple sensors with different uncertainty characteristics. This is a realistic assumption in situations concerning sensor data that are transmitted over communication networks where, generally, multiple sensors with different properties are involved. The uncertainty in each sensor is modelled by a sequence of Bernoulli random variables which are correlated at times and . A real application of such observation model arises, for example, in signal transmission problems where a failure in one of the sensors at time is detected and the old sensor is replaced at time , thus avoiding the possibility of missing signal in consecutive observations.
Using covariance information, the algorithm is derived by applying the innovation technique to suitably defined augmented signal and observation vectors, and the LS quadratic estimator of the signal is obtained from the LS linear estimator of the augmented signal based on the augmented observations.
The performance of the proposed filtering algorithm is illustrated by a numerical simulation example where the state of a first-order autoregressive model is estimated from uncertain observations coming from two sensors with different uncertainty characteristics correlated at times and , considering several values of .
This paper is supported by Ministerio de Educación y Ciencia (Grant no. MTM2008-05567) and Junta de Andalucía (Grant no. P07-FQM-02701).
F. Carravetta and G. Mavelli, “Polynomial filtering for systems with non-independent uncertain observations,” in Proceedings of the 43rd IEEE Conference on Decision and Control, vol. 3, pp. 3109–3114, Atlantis, Bahamas, December 2004.View at: Google Scholar
S. Nakamori, R. Caballero-Águila, A. Hermoso-Carazo, and J. Linares-Pérez, “Linear estimation from uncertain observations with white plus coloured noises using covariance information,” Digital Signal Processing, vol. 138, pp. 552–568, 2003.View at: Google Scholar
J. D. Jiménez-López, J. Linares-Pérez, S. Nakamori, and R. Caballero-Águila, “Signal estimation based on covariance information from observations featuring correlated uncertainty and coming from multiple sensors,” Signal Processing, vol. 88, pp. 2998–3006, 2008.View at: Publisher Site | Google Scholar | Zentralblatt MATH
A. Hermoso-Carazo, J. Linares-Pérez, J. D. Jiménez-López, R. Caballero-Águila, and S. Nakamori, “Recursive fixed-point smoothing algorithm from covariances based on uncertain observations with correlation in the uncertainty,” Applied Mathematics and Computation, vol. 203, no. 1, pp. 243–251, 2008.View at: Publisher Site | Google Scholar | Zentralblatt MATH
J. R. Magnus and H. Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics, John Wiley & Sons, New York, NY, USA, 1999.