Adaptive Complex-Valued Independent Component Analysis Based on Second-Order Statistics

Jia, Yanfei; Yang, Xiaodong

doi:https://doi.org/10.1155/2016/2467198

Journal of Electrical and Computer Engineering

On this page

Abstract Introduction Experimental Results and Analysis Conclusions Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2016 | Article ID 2467198 | https://doi.org/10.1155/2016/2467198

Adaptive Complex-Valued Independent Component Analysis Based on Second-Order Statistics

Yanfei Jia^1,2and Xiaodong Yang^1,3

Academic Editor: Panajotis Agathoklis

Received01 Apr 2016

Revised28 Jul 2016

Accepted16 Aug 2016

Published08 Sept 2016

Abstract

This paper proposes a two-stage fast convergence adaptive complex-valued independent component analysis based on second-order statistics of complex-valued source signals. The first stage constructs a cost function by extending the real-valued whiten cost function to a complex-valued domain and optimizes the cost function using a complex-valued gradient. The second stage uses the restriction that the pseudocovariance matrix of the separated signal is a diagonal matrix to construct the cost function and the geodesic method is used to optimize the cost function. Compared with other adaptive complex-valued independent component analysis, the proposed method shows a faster convergence rate and smaller error. Computer simulations were performed on synthesized signals and communications signals. The simulation results demonstrate the validity of the proposed algorithm.

1. Introduction

Blind source separation (BSS) is the separating of a set of source signals from a set of mixed signals without the aid of information (or with very little information) about either the source signals or the mixing process. Independent component analysis (ICA) is an attractive approach for solving blind source separation problems. ICA can be divided into real-valued ICA and complex-valued ICA according to the mixed signals. Complex-valued ICA is widely used to estimate the mixing matrix or to separate complex-valued mixed signals, such as frequency domain signals [1, 2], digital communication signals [3, 4], functional magnetic resonance imaging signals [5], and power system signals [6].

Studies of complex-valued ICA can be divided into three categories. The first category includes methods based on a nonlinear function, such as complex-valued fastICA (C-fastICA) [7], noncircular complex fastICA (NC-fastICA) [8], complex maximization of non-Gaussiantiy (CMN) [9], complex-valued ICA by entropy bound minimization (CEBM) [10], complex-valued ICA by entropy rate bound minimization (CERBM) [11], and others [3, 12]. The second category includes methods that are based on kurtosis or higher-order cumulants, such as joint approximative diagonalization of eigenmatrix (JADE) [4], kurtosis maximization (KM) [13], pseudo-Euclidean gradient iteration ICA (GEGI-ICA) [14], and others [15–17]. The third category includes methods based on second-order statistics, such as strong-uncorrelating transform (SUT) [18, 19] and its adaptive algorithms [2, 20–22] and pseudo-uncorrelating transform (PUT) [23]. Recently, the performance and separability of complex-valued Gaussian mixtures of SUT method have also been studied [24, 25]. Every complex-valued ICA category has its own merits and appropriate application conditions. The methods based on second-order statistics have a simple structure and low computation complexity and are suitable for complex Gaussian and non-Gaussian noncircular signals. In contrast, ICA methods in the first and second categories are not suitable for use with complex Gaussian noncircular signals.

The major advantage of SUT is that “whenever applicable, remains perhaps the simplest and most accessible approach” [24]. SUT is a batch algorithm and cannot be used to process signals in real time, so some adaptive complex-valued ICA algorithms have been proposed based on second-order statistics [2, 20–22]. Compared with other complex-valued ICA strategies, adaptive complex-valued ICA algorithms based on second-order statistics are simpler in structure and do not require the probability density of the real and imaginary parts of a complex-valued source signal to be non-Gaussian. The Scott method [20] proposes an updating formula of the separating matrix for adaptive complex-valued ICA without mathematical speculation. The Cong method [2] simultaneously uses diagonal covariance and pseudocovariance noncircular signals as the cost function to deduce the adaptive complex ICA. The convergence condition of the Scott and Cong methods requires that the covariance and pseudocovariance of the separated signal are simultaneously diagonal. For example, if only the covariance of the separated signal is diagonal, the method is unable to reach convergence until the pseudocovariance is also diagonal. This requirement could affect convergence speed. The Yang method [22] uses a two-step serial updating method to make the separated signals satisfy the above convergence condition. In the second step, Yang uses the orthogonal method to force the separating matrix to be a unitary matrix. This changes the updating direction of the separating matrix and leads to slow convergence speed.

To increase the rate of convergence, a fast complex-valued ICA method is proposed in this work. The proposed method first extends the real-valued whitening process to a complex-valued domain to provide unit variance for the processed signal. Second, this work uses the restriction that the pseudocovariance matrix of the separated signals is a diagonal matrix to construct cost function and optimize the cost function using the geodesic method. This avoids computing the square root and inverse of the separating matrix and also keeps the separating matrix to be an orthogonal matrix, without any forcing operation. This improves the convergence speed of the proposed method compared to the other adaptive methods.

2. Complex-Valued ICA and Second-Order Statistics

2.1. Complex-Valued Linear ICA Model

Generally, a linear complex-valued ICA model that is noise-free can be expressed as follows:where is the unknown column vector of source signals, is the number of source signals, is the unknown complex-valued mixing matrix, is the column vector of observed complex-valued mixed signals, and is the number of observed signals. The components of the source signals are mutually independent. Most complex-valued ICA algorithms assume that the number of observed signals is not less than the source signals, and only one Gaussian source signal is allowed. The aim of complex-valued ICA is to search the separating matrix and estimate the source signals and mixing matrix. Given that complex-valued ICA does not utilize any information about the source signals or mixing matrix, it has some indeterminacy in amplitude, sequence, and phase. This indeterminacy does not affect the shape of the estimated source signal waveform, which contains most information about source signals.

2.2. Second-Order Statistics of Complex-Valued Signals

Assume a complex-valued random column vector , where and are the real and imaginary part of , respectively, and . The expectation of the random vector is defined as follows:Its covariance matrix is defined as follows:where denotes the Hermitian transpose. Its corresponding pseudocovariance matrix is defined as follows:where denotes the matrix transpose. The covariance matrix together with the pseudocovariance matrix is the full expression of second-order statistics [19]. If the pseudocovariance matrix equals zero, the random vector is considered circular or proper. If both the covariance matrix and pseudocovariance matrix of the random vector are diagonal with nonzero diagonal elements, the random vector is noncircular or improper, and components of the random vector are called strong uncorrelated components.

2.3. Complex-Valued ICA Based on SUT

For any complex random vector , if the vector can be transformed into a random vector by use of a nonsingular square matrix , where has covariance that is a unit matrix and pseudocovariance that is a diagonal matrix with diagonal elements between zero and one, then the matrix is called SUT. If the observed signal is the complex random vector and the source signal is , then the SUT is the separating matrix in complex-valued ICA. The procedure for complex-valued ICA based on SUT is as follows [18].

(1) Whitening the complex-valued observed signals : the whitening procedure is given by where the whitening matrix is the inverse of the matrix square root of the covariance matrix and is the whitened signal with a unit covariance matrix.

(2) Determining the separating matrix of the whitened signal by use of Takagi’s factorization: this is done according toFrom (5) and (6) we obtain the separating matrix .

3. Proposed Adaptive Complex-Valued ICA

In this section, we describe an adaptive fast convergence complex-valued ICA algorithm based on second-order statistics, used in the SUT method. This is unlike other adaptive complex-valued ICA methods that simultaneously force separated signals to comply with second-order statistics. Instead, this method uses an adaptive serial updating method to realize the SUT. First, we use an adaptive method to whiten the observed signals. The cost function used in real-value whitening is directly extended to the complex-valued signal. The cost function is given as follows:where is the whitening matrix and is the th whitening signal. In complex-valued signal processing, the steepest descent direction of cost function (7) iswhere is the observed signal, , , and . To avoid computing the matrix inverse, a complex-valued natural gradient is used to simplify (8):So, adaptive whitening can be expressed as follows:If we use the instantaneous value instead of the expected value in (10), we obtain the adaptive real-time whitening method:Second, we must modify the separated signals to satisfy a diagonal pseudocovariance matrix while keeping the covariance matrix as a unit matrix. We use the cost function in [22], which can be expressed as follows:where , is the separating matrix of the whitened signals and is the diagonal matrix of . The ordinary gradient with is as follows:The update of can be written as follows:where is the correlated matrix of the whitened signal. At the convergence point, the pseudocovariance matrix of the separated signal is diagonal. To keep the covariance matrix of the separated signal as a unit matrix, the separating matrix must be a unitary matrix. In [22], they directly used the method of fixed-point fastICA to force the separating matrix to be a unitary matrix:This approach has two major drawbacks. One is that (16) changes the steepest gradient direction in every iteration, which slows the convergence speed. The second is that (16) must compute the square root and the inverse of the separating matrix in every iteration, which increases the algorithm computation complexity, slowing the time of convergence.

To overcome this problem, we use a geodesic method to search the optimized separating matrix . The geodesic method causes the separating matrix to move on the surface of the orthogonal matrix to converge to a local minimum without a forcing operation. The geodesic method is given bywhere If is a unitary matrix, then is also a unitary matrix. By using the geodesic method, we do not need additional operations to make the separating matrix be an orthogonal matrix and change its search direction.

Using the geodesic method with self-tuning [26] to optimize the cost function (12), we can describe a fast convergence complex-valued ICA method. The implementation process of the proposed adaptive ICA method is as follows:(1)Initialize the whitening matrix and separating matrix using unit matrix, learning rate and , and iterative number for optimizing (7) and (12), respectively.(2)Use (10) to whiten the observed signal and obtain the whitening signal and whitening matrix .(3)Compute the gradient of the cost function in Riemannian space, which can be expressed as follows: where is a diagonal matrix with diagonal elements , , , and .(4)Compute the rotation matrix and .(5)If , , where where and are diagonal matrices corresponding to and , respectively.(6)If , .(7)Update the separating matrix (8)If is sufficiently small, then STOP; else return to step (3).

4. Experimental Results and Analysis

In order to test the algorithm, we used five synthesized signals with different spectral coefficients, three digital communication signals with different spectral coefficients, and three synthesized signals of which two signals have same spectral coefficients as the source signals. For simplicity, we directly used the expectation of the signal instead of the instantaneous value. Quality of separation was assessed using the performance index (PI), a widely used index in ICA. PI can be expressed as [27]where is the element of the global system matrix , is the separating matrix of mixed signal, is the mixing matrix, and and are the maximum absolute value of the elements in the row and column vector , respectively. When perfect separation is achieved, the performance index is zero. “In practice, the value of performance index 10⁻² gives quite a good performance” [27]. The smaller the value of PI, the better the performance.

In the first experiment, five complex-valued synthesized source signals with 10000 samples were used, constructed as follows: where , is a sample drawn from a normal random distribution within , and . The mixing matrix is a complex-valued random matrix with real and imaginary parts generated from a random uniform distribution between 0 and 1. All algorithms have the same learning rate of 0.01 and were run 100 times. Each time, the source signal and mixing matrix was independently generated.

In contrast, convergence curves are shown in Figure 1 that correspond to the four methods: Yang method [22], Scott method [20], SUT method [18], and our proposed method. Every method has 100 convergence curves, and every convergence curve corresponds to results from one run. The SUT method is a batch method without iterative computations. Therefore, the convergence curves are straight lines. From Figure 1, we can see that all the convergence curves of the proposed method are more closer than the other adaptive methods except for the SUT method. This suggests that the proposed method shows improved, stable performance for different mixed sources that is better than the other adaptive methods. The SUT method shows the smallest fluctuation range, followed by the proposed method, Scott method, and then the Yang method. This indicates that the proposed method is more suitable for processing different mixed signals than the other adaptive methods, except for the SUT method. Although the performance of SUT is more stable than the other methods for separating different mixed signals, its realization involves Takagi’s factorization that is difficult to implement and is not suitable for real-time separation of mixed signals. The adaptive complex-valued BSS method is easy to perform and is more appropriate for real-time separation of mixed signals.

Average convergence curves for the four methods are shown in Figure 2. From Figure 2, we see that the Yang method does not converge to a stationary point until 40000 iterations; the Scott method starts to converge close to 35000 iterations; the proposed method starts to converge after about 2000 iterations. Thus, the proposed method has a faster convergence speed. The performance index is larger than the proposed method when the Scott method converges to a stationary point. This indicates that the proposed method has a smaller error than the Scott method. The performance indices of the proposed method and the SUT method are very similar, indicating that the two methods have almost the same amount of error.

In the second experiment, we supposed that three digital communication signals (8QAM, 4QAM, and BPSK) impinge on a uniform linear antenna array with three elements from directions of 10°, 25°, and 70°. In Figure 3, the first row gives the original source signals, the second row gives the three mixed signals that are separately received by the three elements of antenna, and the third row provides the separated signals obtained using the proposed method. Comparing the source signals with the separated signals, we see that the constellation of separated signals is almost the same as the source signals, except the sequence, amplitude, and phase, which are inherently indeterminate. This shows that the proposed method is valid for the supposed communication signals.

The average convergence curves for the four methods are shown in Figure 4 from an average of 100 different simulation runs with a learning rate of 0.01. From Figure 4, we see that the proposed method starts to converge after 150 iterations, the Scott method starts to converge after 4500 iterations, and the slowest to converge is the Yang method, which starts to converge after 17000 iterations. Thus, the proposed method has faster convergence than the other adaptive methods. When the proposed method convergences to the stationary point, the performance index curve of proposed method and SUT method are the same. This means that the two methods have the same error for the communicating signals.

In the third experiment, three random complex-valued signals were used as source signals, with spectral coefficients of 0, 0.6, and 0.6. Their imaginary and real parts were generated by a random uniform distribution function. Average convergence curves from an average of 100 different simulation runs with a learning rate of 0.01 are shown for the four methods in Figure 5. From Figure 5, we see that the performance indexes of Yang method, the Scott method, and the proposed method are less than 0.1 at the stationary point. The average performance index of SUT is about 0.33, which is far greater than 0.1. According to [27], this means that the three adaptive methods successfully separated the mixed signals but the SUT method failed for the mixed signals. The SUT method includes Takagi’s factorization to factorize the pseudocovariance matrix. Therefore, it is not suitable for noncircular signals with the same spectral coefficients.

The proposed method has two stages. The convergence curves shown in all figures are the convergence curves only for the second stage. For the first stage, the whitening signal converges to the unit matrix in first experiment after about 600 iterations and after about 100 iterations in the second and third experiments. Compared with other methods, the total iterations required for the proposed method are far less than other methods.

5. Conclusions

This paper proposes an adaptive complex-valued ICA method for noncircular signals based on second-order statistics and the geodesic method. The proposed method has faster convergence and smaller error than the other adaptive methods. For different mixing source signals, the proposed method has better performance and faster convergence than the Scott method. For source signals with different spectral coefficients, the proposed method and the SUT method have almost the same error. However, the SUT method is not suitable for source signals that some of source signals have the same spectral coefficients.

Competing Interests

The authors declare that they have no competing interests.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (61271115) and the Foundation of Jilin Educational Committee (2015235).

References

S. Nagarajaiah and Y. Yang, “Blind modal identification of output-only non-proportionally-damped structures by time-frequency complex independent component analysis,” Smart Structures and Systems, vol. 15, no. 1, pp. 81–97, 2015.
View at: Publisher Site | Google Scholar
F. Cong, Q.-H. Lin, P. Jia, X. Shi, and T. Ristaniemi, “Second order impropriety based complex-valued algorithm for frequency-domain blind separation of convolutive speech mixtures,” in Proceedings of the 21st IEEE International Workshop on Machine Learning for Signal Processing (MLSP '11), pp. 1–6, Beijing, China, September 2011.
View at: Publisher Site | Google Scholar
G. Qian, P. Wei, and H. Liao, “Efficient variant of noncircular complex fastica algorithm for the blind source separation of digital communication signals,” Circuits, Systems, and Signal Processing, vol. 35, no. 2, pp. 705–717, 2016.
View at: Publisher Site | Google Scholar
J. F. Cardoso and A. Souloumiac, “Blind beamforming for non-gaussian signals,” IEE Proceedings, Part F: Radar and Signal Processing, vol. 140, no. 6, pp. 362–370, 1993.
View at: Publisher Site | Google Scholar
W. Du, G.-S. Fu, V. D. Calhoun, and T. Adah, “Performance of complex-valued ICA algorithms for fMRI analysis: importance of taking full diversity into account,” in Proceedings of the IEEE International Conference on Image Processing (ICIP '14), pp. 3612–3616, Paris, France, October 2014.
View at: Publisher Site | Google Scholar
F. Karimzadeh, S. Esmaeili, and S. H. Hosseinian, “Method for determining utility and consumer harmonic contributions based on complex independent component analysis,” IET Generation, Transmission and Distribution, vol. 10, no. 2, pp. 526–534, 2016.
View at: Publisher Site | Google Scholar
E. Bingham and A. Hyvärinen, “A fast fixed-point algorithm for independent component analysis of complex valued signals,” International Journal of Neural Systems, vol. 10, no. 1, pp. 1–8, 2000.
View at: Publisher Site | Google Scholar
M. Novey and T. Adali, “On extending the complex FastICA algorithm to noncircular sources,” IEEE Transactions on Signal Processing, vol. 56, no. 5, pp. 2148–2154, 2008.
View at: Publisher Site | Google Scholar | MathSciNet
M. Novey and T. Adali, “Complex ICA by negentropy maximization,” IEEE Transactions on Neural Networks, vol. 19, no. 4, pp. 596–609, 2008.
View at: Publisher Site | Google Scholar
X.-L. Li and T. Adali, “Complex independent component analysis by entropy bound minimization,” IEEE Transactions on Circuits and Systems. I. Regular Papers, vol. 57, no. 7, pp. 1417–1430, 2010.
View at: Publisher Site | Google Scholar | MathSciNet
G.-S. Fu, R. Phlypo, M. Anderson, and T. Adalı, “Complex independent component analysis using three types of diversity: non-Gaussianity, nonwhiteness, and noncircularity,” IEEE Transactions on Signal Processing, vol. 63, no. 3, pp. 794–805, 2015.
View at: Publisher Site | Google Scholar | MathSciNet
W. Zhao, Y. Shen, Z. Yuan et al., “A novel method for complex-valued signals in independent component analysis framework,” Circuits, Systems, and Signal Processing, vol. 34, no. 6, pp. 1893–1913, 2015.
View at: Publisher Site | Google Scholar | MathSciNet
H. Li and T. Adali, “A class of complex ICA algorithms based on the kurtosis cost function,” IEEE Transactions on Neural Networks, vol. 19, no. 3, pp. 408–420, 2008.
View at: Publisher Site | Google Scholar
V. James, B. Mikhail, and R. Luis, A Pseudo-Euclidean Iteration for Optimal Recovery in Noisy ICA, vol. 28 of Advances in Neural Information Processing Systems, Montreal Canadiens, 2015.
C. Ji, Y. R. Wang, and X. Y. Wang, “A new complex blind source separation algorithm based on standard kurtosis,” Journal of Northeastern University( Natural Science), vol. 36, no. 5, pp. 614–617, 2015.
View at: Google Scholar | MathSciNet
R.-J. Wang, H.-F. Zhou, Y.-J. Zhan, and M.-Q. Chen, “An algorithm for adaptive complex blind source separation based on Newton update,” Acta Electronica Sinica, vol. 42, no. 6, pp. 1125–1131, 2014.
View at: Publisher Site | Google Scholar
W. Zhao, Y. Wei, Y. Shen et al., “An efficient algorithm by kurtosis maximization in reference-based framework,” Radioengineering, vol. 24, no. 2, pp. 544–551, 2015.
View at: Publisher Site | Google Scholar
J. Eriksson and V. Koivunen, “Complex-valued ICA using second order statistics,” in Proceedings of the IEEE Workshop on Machine Learning for Signal Processing, pp. 183–192, São Luís, Brazil, October 2004.
View at: Google Scholar
J. Eriksson and V. Koivunen, “Complex random vectors and ICA models: identifiability, uniqueness, and separability,” IEEE Transactions on Information Theory, vol. 52, no. 3, pp. 1017–1029, 2006.
View at: Publisher Site | Google Scholar | MathSciNet
C. D. Scott, J. Eriksson, and V. Koivunen, “Equivariant algorithm for estimating the strong-uncorrelating transform in complex independent component analyses,” in Independent Component Analysis and Blind Signal Separation: 6th International Conference, ICA 2006, Charleston, SC, USA, March 5–8, 2006. Proceedings, vol. 3889 of Lecture Notes in Computer Science, pp. 57–65, Springer, Berlin, Germany, 2006.
View at: Publisher Site | Google Scholar
S. C. Douglas, J. Eriksson, and V. Koivunen, “Adaptive estimation of the strong uncorrelating transform with applications to subspace tracking,” in Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, vol. 4, p. 4, Toulouse, France, May 2006.
View at: Publisher Site | Google Scholar
S.-Y. Yang, L.-Q. Zhao, C.-Z. Zhang, and Y.-F. Jia, “Research on adaptive real-time blind separation algorithm of complex mixed signals,” Systems Engineering and Electronics, vol. 31, no. 5, pp. 1018–1021, 2009.
View at: Google Scholar
H. Shen and M. Kleinsteuber, “Algebraic solutions to complex blind source separation,” in Proceedings of the 10th International Conference on Latent Variable Analysis and Source Separation, pp. 74–81, Tel Aviv, Israel, March 2012.
View at: Google Scholar
A. Yeredor, “Performance analysis of the strong uncorrelating transformation in blind separation of complex-valued sources,” IEEE Transactions on Signal Processing, vol. 60, no. 1, pp. 478–483, 2012.
View at: Publisher Site | Google Scholar | MathSciNet
D. Ramírez, P. J. Schreier, J. Vía, and I. Santamaría, “Testing blind separability of complex Gaussian mixtures,” Signal Processing, vol. 95, pp. 49–57, 2014.
View at: Publisher Site | Google Scholar
T. E. Abrudan, J. Eriksson, and V. Koivunen, “Steepest descent algorithms for optimization under unitary matrix constraint,” IEEE Transactions on Signal Processing, vol. 56, no. 3, pp. 1134–1147, 2008.
View at: Publisher Site | Google Scholar | MathSciNet
A. Cichocki and S. I. Amari, Adaptive Blind Signals and Image Processing, John Wiley & Sons, Chichester, UK, 1st edition, 2002.

Copyright

Copyright © 2016 Yanfei Jia and Xiaodong Yang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1652

Downloads

1086

Citations