About this Journal Submit a Manuscript Table of Contents 
Mathematical Problems in Engineering
Volume 2011 (2011), Article ID 150294, 14 pages
doi:10.1155/2011/150294
Research Article

Mixture of Generalized Gamma Density-Based Score Function for Fastica

M. EL-Sayed Waheed, Osama Abdo Mohamed, and M. E. Abd El-aziz

Department of Mathematics, Faculty of Science, Zagazig University, Zagazig 44519, Egypt

Received 14 September 2010; Accepted 21 September 2010

Academic Editor: Ezzat G. Bakhoum

Copyright © 2011 M. EL-Sayed Waheed et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

We propose an entirely novel family of score functions for blind signal separation (BSS), based on the family of mixture generalized gamma density which includes generalized gamma, Weilbull, gamma, and Laplace and Gaussian probability density functions. To blindly extract the independent source signals, we resort to the FastICA approach, whilst to adaptively estimate the parameters of such score functions, we use Nelder-Mead for optimizing the maximum likelihood (ML) objective function without relaying on any derivative information. Our experimental results with source employing a wide range of statistics distribution show that Nelder-Mead technique produce a good estimation for the parameters of score functions.

1. Introduction

By definition, independent component analysis (ICA) is the statistical method that searches for a linear transformation, which can effectively minimize the statistical dependence between its components [1]. Under the physically plausible assumption of mutual statistical independence between these components, the most application of ICA is blind signal separation (BSS). In its simplest form, BSS aims to recover a set of unknown signals, the so-called original sources 𝐬 ( 𝑡 ) = [ 𝑠 1 ( 𝑡 ) , 𝑠 2 ( 𝑡 ) , … , 𝑠 𝑛 ( 𝑡 ) ] 𝑇 ∈ 𝐑 𝑛 , by relying exclusively on information that can be extracted from their linear and instantaneous mixtures 𝐱 ( 𝑡 ) = [ 𝑥 1 ( 𝑡 ) , 𝑥 2 ( 𝑡 ) , … , 𝑥 𝑚 ( 𝑡 ) ] 𝑇 ∈ 𝐑 𝑚 , given by 𝐱 ( 𝑡 ) = 𝐀 𝐬 ( 𝑡 ) , 𝐭 = 1 , 2 , … , 𝑚 , ( 1 . 1 ) where 𝐀 ∈ 𝐑 m x n is an unknown mixing matrix of full rank and 𝑚 ≥ 𝑛 . In doing so, BSS remains truly (blind) in the sense that very little to almost nothing be known a priori for the mixing matrix or the original source signals.

Often sources are assumed to be zero-mean and unit-variance signals with at most one having a Gaussian distribution. The problem of source estimation then boils down to determining the unmixing matrix 𝐖 ∈ 𝐑 n x m such that the linear transformation of the sensor observation is 𝐮 ( 𝑡 ) = 𝐖 𝐱 ( 𝑡 ) , 𝑡 = 1 , 2 , … , 𝑛 , ( 1 . 2 ) where 𝐮 ( 𝑡 ) = [ 𝑢 1 ( 𝑡 ) , 𝑢 2 ( 𝑡 ) , … , 𝑢 𝑛 ( 𝑡 ) ] 𝑇 ∈ 𝐑 𝑛 yield an estimate of vector 𝐬 ( 𝑡 ) corresponding to the original or true sources. In general, the majority of BSS approaches perform ICA, by essentially optimizing the negative log-likelihood (objective) function with respect to the unmixing matrix 𝐖 such that L ( 𝐮 , 𝐖 ) = 𝑛 𝑙 = 1 𝐸 l o g 𝑝 𝑢 l 𝑢 l | | | | − l o g d e t ( 𝐖 ) , ( 1 . 3 ) where 𝐸 [ ⋅ ] represents the expectation operator and 𝑝 𝑢 l ( 𝑢 l ) is the model for the marginal probability density function (pdf) of 𝑢 𝑙 , for all 𝑙 = 1 , 2 , … , 𝑛 . Normally, matrix W is regarded as the parameter of interest and the pdfs of the sources are considered to be nuisance parameters. In effect, when correctly hypothesizing upon the distribution of the sources, the maximum likelihood (ML) principle leads to estimating functions, which in fact are the score functions of the sources [2] 𝜑 𝑙 𝑢 𝑙 𝑑 = − 𝑑 𝑢 𝑙 l o g 𝑝 𝑢 𝑙 𝑢 𝑙 . ( 1 . 4 ) In principle, the separation criterion in (1.3) can be optimized by any suitable ICA algorithm where contrasts are utilized (see; e.g., [2]). A popular choice of such a contrast-based algorithm is the so-called fast (cubic) converging Newton-type (fixed-point) algorithm, normally referred to as FastICA [3], based on 𝐖 𝑘 + 1 = 𝐖 𝑘 𝐸 𝜑 + 𝐃 ( 𝐮 ) 𝐮 𝑇 𝐸 𝜑 − d i a g 𝑙 𝑢 𝑙 𝑢 𝑙 𝐖 𝑘 , ( 1 . 5 ) where, as defined in [4], 1 𝐃 = d i a g 𝐸 𝜑 𝑙 𝑢 𝑙 𝑢 𝑙 𝜑 − E 𝑙 𝑢 𝑙 , ( 1 . 6 ) with 𝜑 ( 𝑡 ) = [ 𝜑 1 ( 𝑢 1 ) , 𝜑 2 ( 𝑢 2 ) , … , 𝜑 𝑛 ( 𝑢 𝑛 ) ] T being valid for all 𝑙 = 1 , 2 , … , 𝑛 . In the ICA framework, accurately estimating the statistical model of the sources at hand is still an open and challenging problem [2]. Practical BSS scenarios employ difficult source distributions and even situations where many sources with very different pdfs are mixed together. Towards this direction, a large number of parametric density models have been made available in recent literature. Examples of such models include the generalized Gaussian density (GGD) [5], the generalized lambda density (GLD), and the generalized beta distribution (GBD) or even combinations and generalizations such as super and generalized Gaussian mixture model (GMM) [6], the generalized gamma density (GGD) [7], the Pearson family of distributions [4], and even the so-called extended generalized lambda distribution (EGLD) which is an extended parameterizations of the aforementioned GLD and GBD models [8]. In the following section, we propose Mixture Generalized Gamma Density (MGΓD) for signal modeling in blind signal separation.

2. Mixture Generalized Gamma Density (MGΓD)

A Mixture Generalized Gamma Density (MGΓD) is a parametric statistical model which assumes that the data originates from weighted sum of several generalized gamma sources [9]. More specifically, a MGΓD is defined as 𝑝 ( 𝑢 ∣ 𝜽 ) = 𝑘 𝑖 = 1 𝑚 𝑖 𝑝 𝑖 𝑢 ∣ 𝑐 𝑖 , 𝛼 𝑖 , 𝛽 𝑖 , 𝛾 𝑖 , ( 2 . 1 ) where(i) 𝜽 = ( 𝑚 𝑖 , 𝑐 𝑖 , 𝛼 𝑖 , 𝛽 𝑖 , 𝛾 𝑖 ) , 𝑖 = 1 , 2 , … , 𝑘 , (ii) 𝑘 is the number of mixture density components,(iii) 𝑚 𝑖 is the 𝑖 th mixture weight and satisfies 𝑚 𝑖 ∑ ≥ 0 , 𝑘 𝑖 = 1 𝑚 𝑖 = 1 ,(iv) 𝑝 𝑖 ( 𝑢 | 𝑐 𝑖 , 𝛼 𝑖 , 𝛽 𝑖 , 𝛾 𝑖 ) is an individual density of the generalized gamma density which is characterized by [7], 𝑝 𝑖 𝑢 ∣ 𝑐 𝑖 , 𝛼 𝑖 , 𝛽 𝑖 , 𝛾 𝑖 = 𝛾 𝑖 𝛽 𝛼 𝑖 𝑖 | | 𝑢 − 𝑐 𝑖 | | 𝛼 𝑖 𝛾 𝑖 − 1 𝛼 2 Γ 𝑖 e x p − 𝛽 𝑖 | | 𝑢 − 𝑐 𝑖 | | 𝛾 𝑖 , ( 2 . 2 ) where 𝑐 𝑖 is the location parameter, 𝛽 𝑖 > 0 is the scale parameter, 𝛼 𝑖 > 0 is the shape/power parameter, and 𝛾 𝑖 > 0 is the shape parameter. Γ ( 𝛼 ) is the gamma function, defined by Γ ( 𝑥 ) = ∞ 0 𝑡 𝑥 − 1 𝑒 − 𝑡 𝑑 𝑡 . ( 2 . 3 ) By varying the parameters, it is possible to characterize a large class of distributions such as Gaussian, sub-Gaussian (more peaked, than Gaussian, heavier tail), and supergaussian (flatter, more uniform). It is noticed that for 𝛾 = 1 , the (GΓD) define gamma density as special case. Furthermore, if 𝛾 = 2 and 𝛼 = 0 . 5 , it become the Gaussian pdf, and if 𝛾 = 1 and 𝛼 = 1 , it represent the Laplacian pdf.

Figures 1 and 2 show some examples of pdf for MGΓD for 𝑘 = 1 and 𝑘 = 2 . Thanks to the shape parameters, the MGΓD is more flexible and can approximate a large class of statistical distributions, this distribution requires to estimate 5 ∗ 𝑘 parameters, 𝜃 ( 𝑚 𝑖 , 𝑐 𝑖 , 𝛼 𝑖 , 𝛽 𝑖 , 𝛾 𝑖 ) 𝑖 = 1 , 2 , … , 𝑘 . Particularly, we discuss the estimation of these parameters in detail in the following section.

150294.fig.001
Figure 1: The probability density function of MGΓD for 𝑘 = 1 , 𝑐 = 3 , 𝛼 = 0 . 5 , 𝛽 = 3 , and 𝛾 = 6 , 2 , 3 . 5 , 1 ( 𝑝 1 , 𝑝 2 , 𝑝 3 , 𝑝 4 ) respectively. The MGΓD allows us to include the case of Gaussian mixture model 𝛼 = 0 . 5 and 𝛾 = 2 .
150294.fig.002
Figure 2: The probability density function of MGΓD for 𝑘 = 2 , 𝑐 1 = 2 , 𝑐 2 = 7 , 𝛼 1 = 𝛼 2 = 0 . 5 , and 𝛽 1 = 𝛽 2 = 2 with different weight 𝑚 1 = 0 . 7 , 0 . 3 , 0 . 4 , 0 . 6 , 𝑚 2 = 1 − 𝑚 1 , and power parameters 𝛾 1 = 1 . 9 , 2 , 2 . 5 , 3 and 𝛾 2 = 2 , 4 , 4 . 3 , 1 . 5 ( 𝑔 , 𝑓 , ℎ , 𝑤 ) , respectively.

3. Numerical Optimization of the Log-Likelihood Function to Estimate MGΓD Parameters

We propose in this section a generalization of the method proposed in [9] which address only the case of 2 components by setting the derivatives of the log-likelihood function to zeros. The log-likelihood function of (2.2), given by [10] 𝐿 ( 𝐮 ∣ 𝜽 ) = 𝑘 𝑖 = 1 𝑁 − 1 𝑗 = 1 ℎ 𝑖 , 𝑗 𝑚 l o g 𝑖 𝑝 𝑖 𝑢 𝑗 ∣ 𝑐 𝑖 , 𝛼 𝑖 , 𝛽 𝑖 , 𝛾 𝑖 , ( 3 . 1 ) where 𝑁 the sample size and ℎ 𝑖 , 𝑗 = 𝑝 ( 𝑖 ∣ 𝑢 𝑗 ) ( 𝑖 = 1 , … , 𝑘 , 𝑗 ∈ [ 0 , 𝑁 − 1 ] ) represents the conditional expectation of 𝑝 𝑖 given the observation 𝑢 𝑗 , this means the posterior probability that 𝑢 𝑗 belongs to the 𝑖 th component. In the case of generalized gamma distribution, if we substitute (2.2) into (3.1) and after some manipulation, we obtain the following form of 𝐿 ( 𝐮 ∣ 𝜽 ) 𝐿 ( 𝐔 ∣ 𝜽 ) = 𝑘 𝑖 = 1 𝑁 𝑗 = 1 ℎ 𝑖 , 𝑗 𝑚 l o g 𝑖 + 𝑘 𝑖 = 1 𝑁 𝑗 = 1 ℎ 𝑖 , 𝑗 𝛾 l o g 𝑖 + 𝛼 𝑖 𝛾 𝑖 | | 𝑢 − 1 l o g 𝑗 − 𝑐 𝑖 | | − l o g ( 2 ) + 𝛼 𝑖 𝛽 l o g 𝑖 Γ 𝛼 − l o g 𝑖 − 𝛽 𝑖 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 . ( 3 . 2 ) Accordingly, we obtain for 𝑖 = 1 , 2 , … , 𝑘 the following nonlinear equation related to the estimated parameters by derivatives of the log-likelihood function with respect to 𝑐 𝑖 , 𝛼 𝑖 , 𝛽 𝑖 , and 𝛾 𝑖 and setting these derivatives to zeros, we obtain 𝜕 𝐿 ( 𝐔 ∣ 𝜽 ) 𝜕 𝑐 𝑖 = 0 , 𝜕 𝐿 ( 𝐔 ∣ 𝜽 ) 𝜕 𝛽 𝑖 = 0 , 𝜕 𝐿 ( 𝐔 ∣ 𝜽 ) 𝜕 𝛾 𝑖 = 0 , 𝜕 𝐿 ( 𝐔 ∣ 𝜽 ) 𝜕 𝛼 𝑖 𝜓 𝛼 = 0 , 𝑖 𝛽 = l o g 𝑖 + ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 l o g 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 ∑ 𝑁 − 1 𝑗 = 1 ℎ 𝑖 𝑗 , 𝛽 𝑖 = 𝛼 𝑖 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝑖 , ( 3 . 3 ) 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 + 𝛼 𝑖 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 l o g 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 = 𝛽 𝑖 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 | | 𝑢 l o g 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 , ( 3 . 4 ) where Ψ ( ⋅ ) is the digamma function ( Ψ ( 𝑥 ) = Γ ( 𝑥 ) / Γ ( 𝑥 ) ) . After a little mathematical manipulation, the ML estimate of 𝛾 𝑖 is obtained 𝜓 ⎛ ⎜ ⎜ ⎝ ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 | | 𝑢 l o g 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 − ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 ∑ 𝑁 𝑘 = 1 ℎ 𝑖 𝑘 | | 𝑢 l o g 𝑘 − 𝑐 𝑖 | | 𝛾 𝑖 ⎞ ⎟ ⎟ ⎠ ⎛ ⎜ ⎜ ⎜ ⎝ ∑ − l o g 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 2 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 | | 𝑢 l o g 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 − ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 ∑ 𝑁 𝑘 = 1 ℎ 𝑖 𝑘 | | 𝑢 l o g 𝑘 − 𝑐 𝑖 | | 𝛾 𝑖 ⎞ ⎟ ⎟ ⎟ ⎠ − ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 l o g 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 = 0 . ( 3 . 5 ) Given the estimate of 𝛾 𝑖 , it is straightforward to derive the estimate for 𝛼 𝑖 , 𝛽 𝑖 , and 𝑐 𝑖 . Let ̂ 𝛾 𝑖 be the estimate of 𝛾 𝑖 . Then, 𝛼 𝑖 = ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | 𝑢 𝑗 − 𝑐 𝑖 | 𝛾 𝑖 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 | | 𝑢 l o g 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 − ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 ∑ 𝑁 𝑘 = 1 ℎ 𝑖 𝑘 | | 𝑢 l o g 𝑘 − 𝑐 𝑖 | | 𝛾 𝑖 , ̂ 𝛽 𝑖 = 𝛼 𝑖 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 ∑ 𝑁 𝑗 = 1 ℎ 𝑖 𝑗 | | 𝑢 𝑗 − 𝑐 𝑖 | | 𝛾 𝑖 , 𝑁 𝑗 = 1 ℎ 𝑖 , 𝑗 𝛼 𝑖 ̂ 𝛾 𝑖 𝜂 𝑢 − 1 𝑗 − ̂ 𝛾 𝑖 ̂ 𝛽 𝑖 𝜂 𝑢 𝑗 | | 𝑢 𝑗 − ̂ 𝑐 𝑖 | | ̂ 𝛾 𝑖 = 0 , ( 3 . 6 ) where 𝜂 ( 𝑥 ) = 1 i f 𝑢 𝑗 − 𝑐 𝑖 < 0 , − 1 i f 𝑢 𝑗 − 𝑐 𝑖 ≥ 0 , ( 3 . 7 ) where 𝛼 𝑖 and ̂ 𝛽 𝑖 are the resulting estimates for 𝛼 𝑖 and 𝛽 𝑖 , respectively, and to estimate the location parameter, we solve (3.4) by gradient ascent. The estimation of weight coefficient obtains directly from ℎ 𝑖 , 𝑗 as follows [10] m 𝑖 = 1 𝑁 N 𝑗 = 1 ℎ 𝑖 , 𝑗 . ( 3 . 8 )

However, (3.5) cannot be easily solved, so we adopt the gradient ascent algorithm to obtain the estimate of 𝛾 𝑖 and determine the estimates of 𝛼 𝑖 , 𝛽 𝑖 , and 𝑐 𝑖 uniquely using this value of 𝛾 𝑖 .

Alternative numerical method can be used to estimate the parameters is called NM, where the appeal of the NM optimization technique lies in the fact that it can minimize the negative of the log-likelihood objective function given in (3.2), essentially without relying on any derivative information. Despite the danger of unreliable performance (especially in high dimensions), numerical experiments have shown that the NM method can converge to an acceptably accurate solution with substantially fewer function evaluations than multidirectional search descent methods. Good numerical performance and a significant improvement in computational complexity for our estimation method, therefore, optimization with the NM technique, produce a good estimation for parameters in MGΓD. To show the performance of NM, we consider the next example.

3.1. Example

We generate random number from MGΓD with parameters 𝑘 = 2 , 𝑚 1 = 0 . 2 5 , 𝑚 2 = 0 . 7 5 , 𝛼 1 = 0 . 5 , 𝛼 2 = 0 . 5 , 𝛽 1 = 2 , 𝛽 2 = 2 , 𝛾 1 = 1 , 𝛾 2 = 4 , 𝑐 1 = 0 , and 𝑐 2 = 1 0 . By performs NM, we obtain best estimation for parameters. As we show in Table 1, the first 5th values of estimated parameters after being sorted according the value of function. In the following section, we resolve to FastICA algorithm for blind signal separation (BSS), this algorithm depends on the estimated parameters and an unmixing matrix W which estimated by FastICA algorithm.

tab1
Table 1

4. Application of MG Γ D in Blind Signal Separation

Novel flexible score function is obtained, by substituting (2.1) into (1.4) for the source estimates 𝑢 𝑙 , 𝑙 = 1 , 2 , … , 𝑛 , it quickly become obvious that our proposed score function inherits a generalized parametric structure, which in turn can be attributed to the highly flexible MGΓD parent model. In this case, a simple calculus yield the flexible BSS score function 𝜑 𝑙 𝑢 l = ∣ 𝜽 − 1 𝑝 𝑢 𝑙 𝑢 𝑙 𝑘 𝑖 = 1 𝑚 𝑖 𝛾 𝑖 𝛽 𝑖 𝛼 𝑖 𝑢 s i g n 𝑙 − 𝑐 𝑖 | | 𝑢 𝑙 − 𝑐 𝑖 | | ( 𝛼 𝑖 𝛾 𝑖 − 2 ) 𝛽 e x p − 𝑖 | | 𝑢 𝑙 − 𝑐 𝑖 | | 𝛾 𝑖 𝛼 2 Γ 𝑖 × 𝛼 𝑖 𝛾 𝑖 − 1 − 𝛽 𝑖 𝛾 𝑖 | | 𝑢 𝑙 − 𝑐 𝑖 | | 𝛾 𝑖 . ( 4 . 1 ) In principle 𝜑 𝑙 ( 𝑢 𝑙 ∣ 𝜽 ) is capable of modeling a large number of signals, such as speech or communication signals, as well as various other types of challenging heavy- and light-tailed distributions.

This is due to the fact that its characterization depends explicitly on all parameters 𝑚 𝑖 , 𝑐 𝑖 , 𝛼 𝑖 , 𝛽 𝑖 , 𝛾 𝑖 , 𝑖 = 1 , 2 , … , 𝑘 . Other commonly used score functions can be obtained by substituting appropriate values for 𝑚 𝑖 , 𝑐 𝑖 , 𝛼 𝑖 , 𝛽 𝑖 , and 𝛾 𝑖 in (4.1), for instance, when 𝑘 = 1 , we have score function 𝜑 𝑙 u l = 𝑢 ∣ 𝜽 s i g n 𝑙 − 𝑐 1 | | 𝑢 𝑙 − 𝑐 1 | | 𝛽 1 𝛾 1 | | 𝑢 𝑙 − 𝑐 1 | | 𝛾 1 − 𝛼 1 𝛾 1 + 1 . ( 4 . 2 ) When 𝛼 1 𝛾 1 = 1 and 𝛽 1 = 1 , we have a scaled form of the GGD-based score function constitutes such a special case of(4.2) 𝜑 𝑙 𝑢 𝑙 ∣ 𝜽 = 𝛾 1 𝑢 s i g n 𝑙 − 𝑐 1 | | 𝑢 𝑙 − 𝑐 1 | | 𝛾 1 − 1 . ( 4 . 3 ) The function 𝜑 𝑙 ( 𝑢 𝑙 ∣ 𝜽 ) could become singular, in some special cases, essentially those corresponding to heavy-tailed (or sparse) distribution defined for 𝛼 𝑖 𝛾 𝑖 ≥ 1 with 𝛼 𝑖 = 1 and | 𝑢 𝑙 − 𝑐 𝑖 | = 0 . In practice, to circumvent such deficiency, the denominator in (4.1) can be modified slightly to read 𝜑 𝑙 𝑢 l = ∣ 𝜽 − 1 𝑝 𝑢 𝑙 𝑢 𝑙 + 𝜀 𝑘 𝑖 = 1 𝑚 𝑖 𝛾 𝑖 𝑢 s i g n 𝑙 − 𝑐 𝑖 | | 𝑢 𝑙 − 𝑐 𝑖 | | ( 𝛼 𝑖 𝛾 𝑖 − 2 ) | | 𝑢 e x p − 𝑙 − 𝑐 𝑖 | | / 𝛽 𝑖 𝛾 𝑖 2 𝛽 𝑖 𝛼 𝑖 𝛾 𝑖 Γ 𝛼 𝑖 × 𝛼 𝑖 𝛾 𝑖 − 𝛾 − 1 𝑖 𝛽 𝛾 𝑖 𝑖 | | 𝑢 𝑙 − 𝑐 𝑖 | | 𝛾 𝑖 , ( 4 . 4 ) where 𝜀 is a small positive parameter (typically around 104) which, when put to use, can almost always guarantee that the discontinuity of (4.1) or values in or approaching the region | 𝑢 𝑙 − 𝑐 𝑖 | = 0 is completely avoided.

5. Numerical Experiments

To investigate the separation performance of the proposed MGΓD-based FastICA BSS method, a set of numerical experiments are preformed, in which we consider only two cases when 𝑘 = 1 , 𝑘 = 2 , and we illustrate this in the following two examples.

5.1. Example  1

In this example, 𝑘 = 1 and the data set used consists of different realizations of independent signals, with distributions shown in Table 2. Note that this is a large-scale and substantially difficult separation problem, since it involves a Gaussian, various super- and sub-Gaussian symmetric PDFs, as well as asymmetric distributions. In all cases, the number of data samples has also been designed to be relatively small; for example, 𝑁 = 2 5 0 . The source signals are mixed (noiselessly) with randomly generated full-rank mixing matrices A . The FastICA method is implemented in the so-called simultaneous separation mode whereas the stopping criterion is set to 𝜀 = 1 0 − 4 . FastICA is executed using the flexible MGΓD model is used to model the distribution of the unknown sources, while (3.3)–(3.5) are employed to adaptively calculate the necessary parameters of the MGΓD-based score function defined in (4.4).

tab2
Table 2

Now, to show the performance in this case, we consider three source signals (source), where these signals are generated randomly from Weilbull, gamma, and exponential distributions as follows: 𝐬 1 𝐬 = w b l r n d ( 1 ∶ 2 5 0 , 1 ) , 2 𝐬 = e x p r n d ( 1 , 1 , 2 5 0 ) , 3 = g a m r n d ( 1 , 1 , 2 5 0 ) . ( 5 . 1 ) Let the mixing matrix A and unmixing matrix W be defined as follows: ⎛ ⎜ ⎜ ⎝ ⎞ ⎟ ⎟ ⎠ ⎛ ⎜ ⎜ ⎝ ⎞ ⎟ ⎟ ⎠ 𝐀 = . 5 6 . 7 9 . 3 7 . 7 5 . 6 5 . 8 6 . 1 7 . 3 2 . 4 8 , 𝐖 = 0 . 2 1 2 8 − 0 . 2 1 7 7 0 . 2 5 6 6 0 . 2 1 8 4 − 0 . 1 9 2 4 0 . 1 6 4 6 1 . 9 4 8 5 − 1 . 8 7 9 8 1 . 8 6 8 7 . ( 5 . 2 ) By using the equation 𝐱 = 𝐀 𝐬 , we obtain mixed signals as show in Figure 3, where mixing signals are in the left and source signals are in the right.

150294.fig.003
Figure 3

After using FastICA, we recover the sources, and we show the estimated signals in the left and original signals in the right in Figure 4 with different in scale only.

150294.fig.004
Figure 4
5.2. Example  2

In this example, 𝑘 = 2 in which the data set used consists of different realizations of independent signals. Note that each signal not only a Gaussian, super-, and sub-Gaussian PDFs, but it is mixed of this PDFs as shown in Table 3.

tab3
Table 3

To show the performance in this case, we consider three source signals (source) where these signals are generated randomly from Gamma_Lapalce ( 𝐬 1 ), Weilbull ( 𝐬 2 ), and Gaussian_Laplace ( 𝐬 3 ) distributions. Let the mixing matrix A and unmixing matrix W be defined as follows: ⎛ ⎜ ⎜ ⎝ ⎞ ⎟ ⎟ ⎠ ⎛ ⎜ ⎜ ⎝ ⎞ ⎟ ⎟ ⎠ 𝐀 = . 5 6 . 7 9 . 3 7 . 7 5 . 6 5 . 8 6 . 1 7 . 3 2 . 4 8 , 𝐖 = − 0 . 1 2 9 3 2 . 6 5 7 9 − 4 . 2 3 2 2 − 1 . 4 4 1 8 − 3 . 1 5 3 3 9 . 9 9 3 0 − 2 . 3 3 1 0 8 . 4 1 0 7 − 1 1 . 6 6 2 7 . ( 5 . 3 ) By using the equation 𝐗 = 𝐀 𝐒 , we obtain mixed signals as show in Figure 5, where mixed signals are in the left and source signals are in the right.

150294.fig.005
Figure 5

After using FastICA, we recover the source, and we show the estimated signals in the left with scale and original signals in the right in Figure 6.

150294.fig.006
Figure 6

6. Algorithm Performance

The separation performance for ICA algorithm is evaluated with the crosstalk error measure 𝑃 𝐼 = 𝑛 𝑖 = 1 𝑛 𝑗 = 1 | | 𝑝 𝑖 𝑗 | | 2 m a x 1 ≤ 𝑙 ≤ 𝑛 | | 𝑝 𝑖 𝑙 | | 2 + − 1 𝑛 𝑗 = 1 𝑛 𝑖 = 1 | | 𝑝 𝑖 𝑗 | | 2 m a x 1 ≤ 𝑙 ≤ 𝑛 | | 𝑝 𝑙 𝑗 | | 2 − 1 . ( 6 . 1 ) Note that here, 𝑝 𝑖 𝑗 represents the elements of the permutation matrix 𝐏 = 𝐖 𝐀 , which after assuming that all sources have been successfully separated should ideally reduce to a permuted and scaled version of the identity matrix. The separation performance for the first example is 𝑃 𝐼 = − 1 2 . 6 8 d B and for second example is 𝑃 𝐼 = − 1 6 . 6 8 d B .

7. Conclusions

We have derived a novel parametric family of flexible score functions, based exclusively on the MGΓD model. To calculate the parameters of these functions in an adaptive BSS setup, we have chosen to maximize the ML equation with the NM optimization method. This alleviates excessive computational cost requirements and allows for a fast practical implementation of the FastICA. Simulation results show that the proposed approach is capable of separating mixtures of signals.

References

  1. P. Comon, “Independent component analysis: a new concept?” Signal Processing, vol. 36, no. 3, pp. 287–314, 1994.
  2. J.-F. Cardoso, “Blind signal separation: statistical principles,” Proceedings of the IEEE, vol. 86, no. 10, pp. 2009–2025, 1998.
  3. A. Hyvärinen and E. Oja, “A fast fixed-point algorithm for independent component analysis,” Neural Computation, vol. 9, no. 7, pp. 1483–1492, 1997.
  4. J. Karvanen and V. Koivunen, “Blind separation methods based on Pearson system and its extensions,” Signal Processing, vol. 82, no. 4, pp. 663–673, 2002. View at Publisher · View at Google Scholar
  5. K. Kokkinakis and A. K. Nandi, “Exponent parameter estimation for generalized Gaussian probability density functions with application to speech modeling,” Signal Processing, vol. 85, no. 9, pp. 1852–1858, 2005. View at Publisher · View at Google Scholar
  6. J. A. Palmer, K. Kreutz-Delgado, and S. Makeig, “Super-Gaussian mixture source model for ICA,” in Proceedings of the International Conference on Independent Component Analysis and Blind Signal Separation, pp. 854–861, Charleston, SC, USA, March 2006.
  7. E. W. Stacy, “A generalization of the gamma distribution,” Annals of Mathematical Statistics, vol. 33, pp. 1187–1192, 1962. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  8. J. Eriksson, J. Karvanen, and V. Koivunen, “Source distribution adaptive maximum likelihood estimation of ICA model,” in Proceedings of the 2nd International Conference on ICA and BSS, pp. 227–232, Helsinki, Finland, June 2000.
  9. G. Dean, “Boerrigter, parameter estimation of the mixed generalized gamma distribution using maximum likelihood estimation and minimum distance estimation,” Presented to the Faculty of the Graduate School of Engineering of the Air Force Institute of Technology .AFIT/GOR/ENS/98M-3.
  10. M. A. T. Figueiredo and A. K. Jain, “Unsupervised learning of finite mixture models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 381–396, 2002. View at Publisher · View at Google Scholar