Adaptive Step Size Gradient Ascent ICA Algorithm for Wireless MIMO Systems

Uddin, Zahoor; Ahmad, Ayaz; Iqbal, Muhammad; Kaleem, Zeeshan

doi:https://doi.org/10.1155/2018/7038531

Mobile Information Systems

On this page

Abstract Introduction Results and Discussion Conclusion Conflicts of Interest References Copyright Related Articles

Research Article | Open Access

Volume 2018 | Article ID 7038531 | https://doi.org/10.1155/2018/7038531

Adaptive Step Size Gradient Ascent ICA Algorithm for Wireless MIMO Systems

Zahoor Uddin,¹Ayaz Ahmad,¹Muhammad Iqbal,¹and Zeeshan Kaleem¹

Academic Editor: Juan C. Cano

Received04 Oct 2017

Accepted18 Mar 2018

Published02 May 2018

Abstract

Independent component analysis (ICA) is a technique of blind source separation (BSS) used for separation of the mixed received signals. ICA algorithms are classified into adaptive and batch algorithms. Adaptive algorithms perform well in time-varying scenario with high-computational complexity, while batch algorithms have better separation performance in quasistatic channels with low-computational complexity. Amongst batch algorithms, the gradient-based ICA algorithms perform well, but step size selection is critical in these algorithms. In this paper, an adaptive step size gradient ascent ICA (ASS-GAICA) algorithm is presented. The proposed algorithm is free from selection of the step size parameter with improved convergence and separation performance. Different performance evaluation criteria are used to verify the effectiveness of the proposed algorithm. Performance of the proposed algorithm is compared with the FastICA and optimum block adaptive ICA (OBAICA) algorithms for quasistatic and time-varying wireless channels. Simulation is performed over quadrature amplitude modulation (QAM) and binary phase shift keying (BPSK) signals. Results show that the proposed algorithm outperforms the FastICA and OBAICA algorithms for a wide range of signal-to-noise ratio (SNR) and input data block lengths.

1. Introduction

Independent component analysis (ICA) [1–3] is a signal processing technique used for estimation [4, 5] of the source signals from the mixed received signals without a priori knowledge of the source signals and the channel condition [3, 6–8]. Before ICA principal component analysis (PCA) [9, 10] was used as a blind source separation (BSS) technique. PCA is based on second-order statistics and makes the data uncorrelated. ICA is based on high-order statistics and makes the data independent. Independence is achieved through maximization of non-Gaussianity. Moreover, ICA has attracted attention from the last two decades due to its effective performance and applicability in practical scenarios [11]. It has various applications in different fields of engineering, that is, vibration analysis [12, 13], robotics [14], machinery fault diagnosis [15, 16], biomedical signal processing [17, 18], speech processing [19], and wireless communication [20, 21]. Its applications in wireless communication systems mainly include multiple input multiple output (MIMO) systems.

The ICA algorithms can be classified into adaptive and batch ICA algorithms [22]. The adaptive algorithms perform well in time-varying mixing scenario with high-computational cost and slow convergence. The batch ICA algorithms have less computational complexity and better separation performance in case of quasistatic mixing scenario. Moreover, amongst the batch algorithms, the gradient ascent ICA (GAICA) algorithm performs well, but step size selection is critical in this algorithm. A smaller value of the step size will produce better results with slow convergence, while a larger step size will result in fast convergence with worse performance of the algorithm. An improper choice of the step size will produce worse performance or demand for large iterations [23].

In the literature, various step size selection techniques are presented [24–27], where all these techniques are based on the fact that a smaller step size produces better results with slow convergence and larger step size provides fast convergence with worse performance. In [24], a so-called modified gradient-based ICA algorithm is developed. This algorithm converges for various values of the step size parameter if the objective function is properly selected. The authors in [25] presented a gradient ascent ICA algorithm, where the algorithm is terminated when the relative change in the unmixing matrices becomes less than a certain defined value for various step sizes. Furthermore, a diminishing step size gradient-based ICA technique is proposed in [26, 27], where the step size is adapted in such a way that its initial value is high and which is repeatedly reduced with increasing iterations in order to converge to the optimal value.

In this paper, an adaptive step size gradient ascent ICA (ASS-GAICA) algorithm is developed. The proposed algorithm is free from the selection of the step size parameter. The objective function used for maximization of the non-Gaussianity in our algorithm is an absolute value of entropy [6]. The basic principle used in the derivation of the algorithm is that entropy of the recorded mixed signals is always less than the total entropy of the independent source signals and maximization of entropy produces the separated source signals. In addition to this, different performance evaluation criteria are used to observe the effectiveness of the proposed algorithm. These criteria include signal-to-interference ratio (SIR), symbol error rate (SER), error standard deviation (ESTD) of the signal, and norm of the product of mixing and unmixing matrices (Norm-WA). It is shown in the simulation that performance of the proposed algorithm is better than the fixed step size GAICA algorithm. Performance of the proposed ASS-GAICA algorithm is also compared with the well-known ICA algorithms: the FastICA [19] and the OBAICA algorithms [28].

Notations used in this paper are , , and for transpose, derivative, and second-order derivative of a vector or a matrix, respectively. Capital bold face letters represent matrices, and the small bold face letters are used for vectors. Symbols and abbreviations used in this paper are listed in Tables 1 and 2, respectively.

Rest of the paper is organized as follows: Section 2 presents the ICA signals’ model and assumptions. The ICA model is purely based on multidimensional data. This section presents the basic idea that how to model the MIMO signals for ICA algorithm. The mathematical formulation of the proposed ASS-GAICA algorithm is demonstrated in Section 3. Section 4 illustrates the simulation results. The results are demonstrated for quasistatic and time-varying mixing scenario utilizing various performance evaluation criteria. Finally, the conclusion is drawn in Section 5.

2. ICA Signals’ Model and Assumptions

A multiuser multiple input multiple output (MIMO) [29] wireless communication system with transmitting antennas and receiving antennas is considered. For the sake of simplicity, we have . The transmitted source signal vectors are , where and the received mixed signals are . The block diagram of the system model is shown in Figure 1. The ICA postprocessed estimated signals are . The role of the ICA algorithms is to unmix the received mixed signals in order to find the actual transmitted signals [3]. The received mixed signals can be modeled in matrix form as follows:where is mixed data matrix, is mixing matrix, and is source data matrix. Equation (1) represents the noise free model of ICA. The noisy ICA model can be represented aswhere represents the additive white Gaussian noise with dimension . The addition of noise further degrades the separation performance of the ICA algorithm. Moreover, it is known that inverse of the mixing matrix exist which is known as the unmixing matrix and denoted by . The role of the ICA algorithms is to estimate the unmixing matrix. After estimating the unmixing matrix, the resultant unmixed signals can be modeled as follows:

After estimating the unmixing matrix , the mixing source signals can be separated from the mixed signals by using (3). Accuracy of the ICA algorithms depends on proper estimation of the unmixing matrix. In case, if the mixing matrix changes dynamically within the processing data blocks, then unmixing becomes difficult.

The basic assumption of ICA used in this paper is given below:(i)The source signals are mutually statistically independent. The transmitting antennas transmit different source signals, and thus the independent assumption becomes true.(ii)The source signals are non-Gaussian in nature. This assumption is also true because we consider the communication signals which are non-Gaussian in nature.(iii)The mixing matrix is assumed to be square. The validity of this assumption can be observed from the equal number of the transmitting and receiving antennas.

3. Proposed ASS-GAICA Algorithm

The absolute value of entropy is used as an objective function in the derivation of the algorithm. The symbols used in the formulation of the algorithm are listed in Table 1. Furthermore, it is known from the ICA model that the source signals have non-Gaussian distributions. The CDF and PDF of the transmitted source signals are represented by and , respectively. It is considered in [22] that the PDF of the supper-Gaussian signal can be represented by and sub-Gaussian by . Received signals are the linear mixtures of the transmitted source signals, and the source signals can be estimated from the mixed received signals through the ICA algorithm. The ICA algorithm basically estimates the unmixing matrix [3]. General form of the gradient ascent ICA algorithm is given in Algorithm 1. Hence, entropy of the estimated signals for iteration can be written in vector matrix form as

(1)	Initialization
	Whiten the mixed received data
	Choose a random initial unmixing matrix
	Select the number of maximum iterations
(2)	for to do
(3)
(4)	Update the unmixing matrix using the following equation:

(5)	Terminate the loop if
	, where is very small
(6)	end for

Maximization of entropy as given in (4) tends toward maximization of the non-Gaussianity. In order to achieve this objective, the gradient of with respect to each element of can be written aswhere

Now, general form of the update equation for can be written as

After substituting (5) in (7), we will get the following updated formula for :

Equation (8) represents the fixed step size GAICA algorithm where the positive sign used in (8) indicates the gradient ascent nature of the algorithm. This equation updates the unmixing matrix in such a way to further maximize the entropy while utilizing a constant step size. The optimal step size selection is difficult in this equation. The improper value of will result in worse performance or slow convergence of the algorithm. Moreover, in order to achieve the adaptive step size mechanism, first determine at iteration. Then, approximate up to the second-order terms by using Taylor’s series expansion. Finally, adjust the value of in such a way to further maximize the entropy while processing from iteration to iteration. In order to achieve these objectives first, we determine the entropy for iteration as

After applying Taylor’s series expansion over (9), the resultant equation can be written aswhere .

For smaller values of , can be approximated up to second-order terms in (10), and this is indeed the case in our algorithm, which is shown in the simulation section. Using this fact in combination with (5), we can write (10) as

From (8), we have

After combining (12) and (11), we get

In order to achieve the adaptive step size selection criterion, adjust in such a way to further maximize . To achieve this objective, evaluate the gradient of with respect to as

Now after combining (13) and (14), we get the following resultant equation:

Now substituting (15) in (8), we have

Letthen (16) can be written as

The updated formula (18) involves inversion of the denominator part of , that is, , having same dimension as matrix . It means that the proposed algorithm is more suitable for a smaller number of the source signals because the computational complexity of the matrix inversion is . In case of large , an estimate of can be developed. Another factor effecting the computational complexity is the data block length , and hence computational complexity of the proposed algorithm is that is approximately similar to the computational complexity of the fixed step size GAICA algorithm. The proposed algorithm is summarized in Algorithm 2. We use as a stopping criterion in our algorithm.

(1)	Initialization
	Whiten the mixed received data
	Choose a random initial unmixing matrix
	Select the number of maximum iterations
(2)	for to do
(3)
(4)	Calculate entropy of the mixed received data vectors as

(5)	Calculate matrix as follows:

(6)	Calculate matrix as

(7)	Update the unmixing matrix using the following equation:

(8)	Terminate the loop if
	, where is very small
(9)	end for

4. Results and Discussion

Performance of the proposed ASS-GAICA algorithm is demonstrated in this section. A dual antenna MIMO transceiver system is considered for simulations. The receiver receives linear mixtures of the transmitted signals. The channel is assumed AWGN with frequency flat fading. Simulation is performed over BPSK and 16-QAM signals with SNR ranging from −2 dB to 10 dB and input data block lengths ranging from 50 to 1000 samples using Monte–Carlo simulation. We consider the quasistatic and the time-varying wireless channels. The simulation parameters are shown in Table 3.

The order, phase, and amplitude indeterminacies are resolved by using a technique presented in [30], where a few extra samples are transmitted which are known to the receiver in advance. This technique is insensitive to the data block lengths.

Different performance evaluation criteria are used to measure the effectiveness of the proposed algorithm which is given below.(1)Signal-to-interference ratio (SIR): The SIR represents the average ratio of the desired signal power to the power of the estimation error. The SIR in dB can be written as(2)Error standard deviation (ESTD) of the signal: Error signal is the difference of the source signal and the algorithm postprocessed signal. The ESTD can be defined aswhere .(3)Symbol error rate (SER).(4)Norm of the product of mixing and unmixing matrices (Norm-WA): For ideal case the norm of the product of and will be one because . A value of Norm-WA near to one represents efficient separation. Mathematically it can be defined as follows:

Simulation results for quasistatic wireless channels are demonstrated as follows.

First, we demonstrate the convergence behavior of the proposed ASS-GAICA algorithm and the fixed step size GAICA algorithm. The GAICA is an algorithm where the step size can be adjusted experimentally. The smaller value of the step size improves the separation performance with slower convergence, while the larger value increases the convergence speed with less accurate results. In order to compare the results of the GAICA algorithm with our proposed algorithm, we consider two values of the step size parameter, that is, . The results are shown in Figure 2 for 16-QAM signals with and SNR of 10 dB. We consider the maximum number of iterations as 50 for both the algorithms. The step size values considered for the GAICA algorithm are . It can be observed from the results that further decrease in the step size parameter will further reduce the convergence speed. Similarly, increase in the step size will produce worse performance of the algorithm. Hence, we restrict ourselves to these two values of the step size parameter. The proposed algorithm properly converges in 22 iterations, while the GAICA algorithm converges in 40 iterations for . If we increase the value of to 0.2, then the GAICA algorithm converges faster with less accurate steady state results in 50 iterations. In summary, we can say that the GAICA algorithm converges faster for larger step size with less accurate steady state results and vice versa, while the proposed algorithm improves the convergence speed as well as quality of the results.

Secondly, the performance of the proposed algorithm is compared with the well-known algorithm of ICA, FastICA algorithm and OBAICA algorithm using different performance measures as a cross-check. Figures 3 and 4 show the effectiveness of the proposed algorithm in terms of the SIR for different block lengths and different SNRs using 16-QAM signals. In Figure 3, we consider the fixed SNR of 10 dB with different block lengths, while in Figure 4, the block length is kept 1000 samples with different values of SNR.

The performance of the proposed algorithm is again evaluated utilizing the error standard deviation (ESTD) of the signal. The results are shown in Figures 5 and 6 for different blocks lengths and SNR using 16-QAM signals. The Norm-WA and SER performance are shown in Figures 7 and 8. All these figures show the superior performance of the proposed ASS-GAICA algorithm. From Figures 3–8 and Table 4, one can observe that the FastICA algorithm performs well than the OBAICA algorithm for the observed signals, but accuracy of the results is high for the proposed ASS-GAICA algorithm.

Another experiment is performed while utilizing the BPSK-modulated signals. The number of source signals and the simulation environment are the same as above. The SIR is used as a performance measure. The SIR performance for different block lengths and fixed SNR is shown in Figure 9. This figure shows that the proposed algorithm outperforms the FastICA and the OBAICA algorithms for quasistatic channels.

We also investigate the proposed algorithm in comparison with the FastICA and the OBAICA algorithms for time-varying mixing scenario. The time-varying mixing matrix is represented as follows:where , , , and are the random variables representing the time-varying nature of the wireless channels. The results for time-varying mixing scenario are shown in Table 5. The results given in this table show that even at time-varying scenario our proposed algorithm outperforms.

5. Conclusion

The entropy-based gradient ascent ICA (GAICA) algorithm has improved accuracy of separation, but the step size selection is difficult in this algorithm. Improper step size will result in worse performance of the algorithm. In this paper, we proposed an adaptive step size gradient ascent ICA (ASS-GAICA) algorithm that selects the optimal step size adaptively. The proposed algorithm has better convergence behavior with improved accuracy of results than the fixed step size GAICA algorithm. Performance of the proposed algorithm is compared with the well-known ICA algorithms, the FastICA and the OBAICA algorithms using binary phase shift keying (BPSK) and quadrature amplitude modulation (QAM) signals in quasistatic and time-varying scenario. Different performance measures, that is, SIR, ESTD, SER, and Norm-WA, are used to evaluate the effectiveness of the proposed algorithm. The simulation results show that the proposed algorithm outperforms the FastICA and the OBAICA algorithms for various input data blocks and SNR. The proposed algorithm outperforms even for time-varying scenario.

Due to matrix inversion in the update equation, the proposed algorithm is suitable for a smaller number of the mixing source signals. In case of a large number of the mixing signals, some estimate of the matrix is required. As a future work, we will develop an estimator of this matrix for large number of the source signals.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

References

J. R. Wessel, “Testing multiple psychological processes for common neural mechanisms using EEG and independent component analysis,” Brain Topography, vol. 31, no. 1, pp. 90–100, 2016.
View at: Publisher Site | Google Scholar
Z. Uddin, A. Ahmad, M. Altaf, and F. Alam, “Gradient ascent independent component analysis algorithm for telecommunication signals,” Journal of Engineering and Applied Sciences (JEAS), vol. 36, no. 1, pp. 125–133, 2017.
View at: Google Scholar
Z. Uddin, A. Ahmad, and M. Iqbal, “ICA based MIMO transceiver for time varying wireless channels utilizing smaller data blocks lengths,” Wireless Personal Communications, vol. 94, no. 4, pp. 3147–3161, 2017.
View at: Publisher Site | Google Scholar
S. He, Y. Wang, and C. Liu, “Modified partial least square for diagnosing key-performance-indicator-related faults,” Canadian Journal of Chemical Engineering, vol. 96, no. 2, pp. 444–454, 2018.
View at: Publisher Site | Google Scholar
Y. Wang, D. Zhao, Y. Li, and S. X. Ding, “Unbiased minimum variance fault and state estimation for linear discrete time-varying two-dimensional systems,” IEEE Transactions on Automatic Control, vol. 62, no. 10, pp. 5463–5469, 2017.
View at: Publisher Site | Google Scholar
J. V. Stone, Independent Component Analysis: A Tutorial Introduction, MIT Press, Cambridge, UK, 2004.
G. S. Khorshidia, G. Douauda, C. F. Beckmannb, M. F. Glasserd, L. Griffantia, and S. M. Smitha, “Automatic denoising of functional MRI data combining independent component analysis and hierarchical fusion of classifiers,” NeuroImage, vol. 90, pp. 449–468, 2014.
View at: Publisher Site | Google Scholar
Z. Ge and Z. Song, “Ensemble independent component regression models and soft sensing application,” Chemometrics and Intelligent Laboratory Systems, vol. 130, pp. 115–122, 2014.
View at: Publisher Site | Google Scholar
Z. Lou, D. Shen, and Y. Wang, “Two-step principal component analysis for dynamic processes monitoring,” Canadian Journal of Chemical Engineering, vol. 96, no. 1, pp. 160–170, 2018.
View at: Publisher Site | Google Scholar
Z. Lou and Y. Wang, “Multimode continuous processes monitoring based on hidden semi-Markov model and principal component analysis,” Industrial and Engineering Chemistry Research, vol. 56, no. 46, pp. 13800–13811, 2017.
View at: Publisher Site | Google Scholar
J. Y. Zhang, J. An, and H. Zhang, “Monitoring of time-varying processes using kernel independent component analysis,” Chemical Engineering Science, vol. 88, pp. 23–32, 2013.
View at: Publisher Site | Google Scholar
H. Razaghi, R. Saatchi, A. Offiah, D. Burke, N. Bishop, and S. Gautam, “Assessing material densities by vibration analysis and independent component analysis,” Malaysian Journal of Fundamental and Applied Sciences, vol. 9, no. 3, pp. 123–128, 2013.
View at: Publisher Site | Google Scholar
X. Li, W. Li, Y. Sun, and H. Zhao, “Blind source separation of vibration signal of electric traction experiment system,” in Proceedings of the IEEE International Conference on Intelligent Control and Information (ICICIP), pp. 93–96, Dalian, China, August 2010.
View at: Google Scholar
H. Saruwatari, Y. Mori, T. Takatani et al., “Two-stage blind source separation based on ICA and binary masking for real-time robot audition system,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2303–2308, Edmonton, AB, Canada, August 2005.
View at: Google Scholar
W. Lou, G. Shi, and J. Zhang, “Research and application of ICA technique in fault diagnosis for equipments,” in Proceedings of the IEEE International Conference on Intelligent Computing and Intelligent Systems, vol. 4, pp. 310–313, Shanghai, China, November 2009.
View at: Google Scholar
J. Fan and Y. Wang, “Fault detection and diagnosis of non-linear non-Gaussian dynamic processes using kernel dynamic independent component analysis,” Journal of Information Sciences, vol. 259, pp. 369–379, 2014.
View at: Publisher Site | Google Scholar
H. Mehrabian, R. Chopra, and A. Martel, “Calculation of intravascular signal in dynamic contrast enhanced-MRI using adaptive complex independent component analysis,” IEEE Transactions on Medical Imaging, vol. 32, no. 4, pp. 699–710, 2013.
View at: Publisher Site | Google Scholar
M. Boiret, D. N. Rutledge, N. Gorretta, Y. Ginot, and J. Roger, “Application of independent component analysis on raman images of a pharmaceutical drug product: pure spectra determination and spatial distribution of constituents,” Journal of Pharmaceutical and Biomedical Analysis, vol. 90, pp. 78–84, 2014.
View at: Publisher Site | Google Scholar
Z. Uddin and F. Alam, “Hardware implementation of blind source separation of speech signals using independent component analysis,” International Journal of Electrical and Computer Sciences, vol. 10, pp. 85-86, 2010.
View at: Google Scholar
D. Obradovic, N. Madhu, A. Szabo, and C. S. Wong, “Independent component analysis for semi-blind signal separation in MIMO mobile frequency selective communication channels,” in Proceedings of the IEEE International Joint Conference on Neural Networks, vol. 1, pp. 53–58, Budapest, Hungary, July 2004.
View at: Google Scholar
M. Gupta and B. Santhanam, “ICA based blind adaptive MAI suppression in DS-CDMA systems,” in Proceedings of the 3rd IEEE Signal Processing Education Workshop, pp. 201–205, Taos Ski Valley, NM, USA, August 2004.
View at: Google Scholar
X. Tan, X. Zhang, and J. Ye, “Adaptive step size natural gradient ICA algorithm with weighted orthogonalization,” Circuits Systems and Signal Processing, vol. 33, no. 1, pp. 211–221, 2014.
View at: Publisher Site | Google Scholar
S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cambridge, UK, 2009.
H.-M. Park, S.-H. Oh, and S.-Y. Lee, “A modified infomax algorithm for blind signal separation,” Neurocomputing, vol. 70, no. 1–3, pp. 229–240, 2006.
View at: Publisher Site | Google Scholar
X. Wangn, Z. Huang, Y. Zhou, and X. Ren, “Approaches and applications of semi-blind signal extraction for communication signals based on constrained independent component analysis: the complex case,” Neurocomputing, vol. 101, pp. 204–216, 2013.
View at: Publisher Site | Google Scholar
Q. Jin and G. Liang, “A new algorithm of infomax for small numbers of sound signal separation,” in Proceedings of the International Conference on Artificial Intelligence and Education (ICAIE), pp. 159–162, Hangzhou, China, October 2010.
View at: Google Scholar
Q. Jin, G. Wang, and Y. Liu, “Blind signal separation by entropy maximization (infomax),” in Proceedings of the 6th International Conference on Wireless Communications Networking and Mobile Computing (WiCOM), pp. 1–5, Chengdu, China, September 2010.
View at: Google Scholar
W. B. Mikhael and T. Yang, “A gradient-based optimum block adaptation ICA technique for interference suppression in highly dynamic communication channels,” EURASIP Journal on Advances in Signal Processing, vol. 2006, pp. 1–10, 2006.
View at: Publisher Site | Google Scholar
Z. Kaleem, B. Hui, and K. H. Chang, “Achievable rates of SVD-based codebooks for zero-forcing and Tomlinson-Harashima precoding schemes with limited feedback MU-MIMO system,” International Journal of Communication Systems, vol. 30, no. 7, p. e3154, 2017.
View at: Publisher Site | Google Scholar
Y. Jiang, X. Zhu, E. Lim, L. Dong, and Y. Huang, “Low-complexity independent component analysis based semi-blind receiver for wireless multiple-input multiple-output systems,” International Journal of Design, Analysis and Tools for Circuits and Systems, vol. 2, pp. 91–98, 2011.
View at: Google Scholar

Copyright

Copyright © 2018 Zahoor Uddin et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1660

Downloads

974

Citations