Abstract

The alternating direction method of multipliers (ADMM) has been widely explored due to its broad applications, and its convergence has been gotten in the real field. In this paper, an ADMM is presented for separable convex optimization of real functions in complex variables. First, the convergence of the proposed method in the complex domain is established by using the Wirtinger Calculus technique. Second, the basis pursuit (BP) algorithm is given in the form of ADMM in which the projection algorithm and the soft thresholding formula are generalized from the real case. The numerical simulations on the reconstruction of electroencephalogram (EEG) signal are provided to show that our new ADMM has better behavior than the classic ADMM for solving separable convex optimization of real functions in complex variables.

1. Introduction

The augmented Lagrangian methods (ALMs) are a certain class of algorithms for solving constrained optimization problems, which were originally known as the method of multipliers in 1969 [1], and were studied much in the 1970s and 1980s as a good alternative to penalty methods. They have similarities to penalty methods in that they replace a constrained optimization problem by a series of unconstrained problems and add a penalty term to the objective. In particular, a variant of the standard ALMs that uses partial updates (similar to the Gauss-Seidel method for solving linear equations) known as the alternating direction method of multipliers (ADMM) gained some attention [2]. The ADMM has been extensively explored in recent years due to broad applications and empirical performance in a wide variety of problems such as image processing [3], applied machine learning and statistics [4], sparse optimizations, and other relevant fields [2]. Specifically, an advantage of the ADMM is that it can handle linear equality constraint of the form , which makes distributed optimization by variable splitting in a batch setting straightforward. Recently, the convergence rates of order for the real case are considered under some additional assumptions; see, for example, [5–10]. For a survey on the ALMs and the ADMM, we refer to the references [1, 2, 11–16].

Compressed sensing (CS) is a signal processing technique for efficiently acquiring and reconstructing a signal by finding solutions to underdetermined linear systems. In the CS processing, the sparsity of a signal can be exploited to recover it from samples far fewer than required by the Shannon-Nyquist sampling theorem. The idea of CS got a new life in 2006 when Candès et al. [17] and Donoho [18] gave important results on the mathematical foundation of CS. This methodology has attached much attention from applied mathematicians, computer scientists, and engineers for a variety of applications in biology [19], medicine [20], and radar [21], and so forth. Algorithms for signal reconstruction in a CS framework are expressed as sparse signal reconstruction algorithms. One of the most successful algorithms, known as basis pursuit (BP), is on the basis of constrained -norm minimization [22]. Most of the work is focused on the optimization in the real number field.

Signals in complex variables emerge in many areas of science and engineering and have become the objects of signal processing. There have been many works on the signal processing in complex variable. For example, independent component analysis (ICA) for separating complex-valued signals has found utility in many applications such as face recognition [23], analysis of functional magnetic resonance imaging [24], and electroencephalograph [25]. Taking impropriety and noncircularity of complex-valued signals into consideration, the right type of processing can give significant performance gains [26]. Methods of digital modulation schemes that produce improper complex signals have been studied in [27], such as binary phase shift keying and pulse amplitude modulation. In these researches, most nonlinear optimization methods use the first-order or second-order approximation of the objective function to create a new step or a descent direction, where the approximation is either updated or recomputed in every iteration. Unfortunately, all these functions do not satisfy the Cauchy-Riemann conditions. There exists no Taylor series of at so that the series converges to in a neighborhood of . A common solution is to convert the optimization problem to the real domain by referring to as a function of the real and imaginary parts of . Reformulating an optimization problem which is inherently complex to the real domain would miss important insights on the structure of the problem that might otherwise be exploited [28]. Even so, there are many primal-dual optimization methods for optimization problems with the complex variable in the literatures. The usual method analyzing complex-valued optimization problem is to separate it into the real part and the imaginary part and then to recast it into an equivalent real-valued optimization problem by doubling the size of the constraint conditions; see [29–31] and the references therein. For some other related references on optimization problems in complex variables, we refer to [28, 32].

To overcome the above-mentioned difficulties, the purpose of the paper is to generalize ADMM for separable convex optimization in the real number domain to the complex number domain. The concepts of convex function and subgradient are expanded from the real field to the complex field. By introducing the augmented complex variable, the definition of the augmented Lagrange function in complex variables is given. Under some mild assumptions, we establish the convergence of the proposed method. For the applications, we consider the BP algorithm which concludes projection algorithm and the soft thresholding operator in the complex field. Some numerical simulation results are reported to show that the proposed algorithm is indeed more efficient and more robust.

The outline of the paper is as follows. In Section 2, we recall some elementary theories and methods of the complex analysis and Wirtinger calculus. The ADMM for complex separable convex optimization and its convergence are presented in Section 3. In Section 4, we study the BP algorithm for the equality-constrained minimization problem in the form of ADMM. In Section 5, some numerical simulations are provided. Finally, some conclusions are drawn in Section 6.

2. Preliminaries

In this section, we first give some notations used. Vectors are denoted by lower case, for example, , and matrices are denoted by capital letters, for example, . The th entry of a vector is denoted by and element of a matrix by . The subscripts and denote the real and imaginary parts, respectively; for example, and . The superscripts and are used for the transpose, conjugate, Hermitian conjugate, and matrix inverse. The dom denotes the domain of function . The identity matrix of order is denoted by . The one-norm and two-norm are denoted by and , respectively. denotes the real composite -dimensional vector; for example, , obtained by stacking on the top of . The notation denotes the set of all subgradients of .

2.1. Wirtinger Calculus

We next recall some well-known concepts and results on the complex analysis and Wirtinger calculus which will be used in our future analysis. A comprehensive treatment of Wirtinger calculus can be found in [33, 34].

Define the complex augmented vector as follows: which is obtained by stacking on the top of its complex conjugate . The complex augmented vector is related to the real composite vector as and , where the real-to-complex transformationis unitary up to a factor of 2; that is, . The linear map is an isomorphism map from to and its inverse is given by .

Lemma 1. Let , , and . Thenwhere

Proof. Sincethen we haveThis completes the proof.

Consider a complex-valued function where , , and . The definition of complex differentiability requires that the derivatives be defined as the limit is independent of the direction in which approaches zero in the complex plane. This requires that the Cauchy-Riemann equations should be satisfied [35]. These conditions are necessary for to be complex-differentiable. A function which is complex-differentiable on its entire domain is called analytic or holomorphic. Clearly, the Cauchy-Riemann conditions do not hold for real-valued functions which are , and thus cost functions are not analytic. These conditions imply complex differentiability which are quite stringent and impose a strong structure on and and, consequently, on . Obviously, most cost functions do not satisfy the Cauchy-Riemann equations as these functions are typically with .

To overcome such a difficulty, a sound approach in [33] relaxes this strong requirement for differentiability and defines a less stringent form for the complex domain. More importantly, it describes how this new definition can be used for defining complex differential operators that allow computation of derivatives in a very straightforward manner in the complex number domain, by simply using real differentiation results and procedures. A function is called real differentiable when and are differentiable as the functions of real-valued variables and . Then, one can write the two real variables as and and use the chain rule to derive the operators for differentiation given in the theorem below. The key point in the derivation is regarding the two variables and as independent variables, which is also the main approach allowing us to make use of the elegance of Wirtinger calculus.

In view of this, we consider the function (7) as by rewriting it as and make use of the underlying structure. The function can be regarded as either with variables and or with variables and , and it can be simply written as . The functions may take different forms; however, they are equally valued. For convenience, we use the same function to denote them as follows: The main result in this context is stated by Brandwood in [36].

Theorem 2. Let be a function of real variables and such that , where , and that is analytic with respect to and independently. Then, consider the following:
(1) The partial derivatives can be computed by treating as a constant in and as a constant, respectively.
(2) A necessary and sufficient condition for to have a stationary point is that Similarly, is also a necessary and sufficient condition.

As for the applications of Wirtinger derivatives, we consider the following two examples, which will be used in the subsequent analysis.

Example 3. Consider the real function in complex variables as follows: where , , , , , and .

It follows from Theorem 2 that Similarly, we have

Example 4. Consider the real function in complex variables as follows: where , , , , and .

We have where . Then

2.2. Convex Analysis in the Complex Number Domain

In order to meet the demands of next work, we give some definitions in the complex number domain.

Definition 5 (see [37]). A set is convex if the line segment between any two points in lies in ; that is, if for any and any with , then

Definition 6 (see [34]). Let . The complex gradient operator is defined by

The linear map also defines a one-to-one correspondence between the real gradient and the complex gradient ; namely,

For real function in complex variable , it has an equivalent form as according to (10). So we can similarly extend some concepts of the functions in the real number domain [38, 39] to the complex number domain.

Definition 7. A real function in complex variable is convex if is a convex set and if for any and any with , then

Definition 8. A real function in complex variable is proper if its effective domain is nonempty and it never attains .

Definition 9. A real function in complex variable is closed if, for each , the sublevel set is a closed set.

Definition 10. Given any real function in complex variable , a vector is said to be a subgradient of at if

3. ADMM for Convex Separable Optimization

In this section, we will first recall the ADMM for real convex separable optimization. Then we will study the ADMM for convex separable optimization of real functions in complex variables.

3.1. ADMM for Real Convex Separable Optimization

The ADMM has been well studied for the following linearly constrained separable convex programming whose objective function is separated into two individual convex functions with nonoverlapping variables as follows: where and are closed convex sets; and are given matrices; is a given vector; and and are proper, closed, and convex functions.

More specifically, the Lagrangian function and the augmented Lagrangian function of (25) are given by respectively. Then the iterative scheme of the ADMM for solving (25) is given byWithout loss of generality, we give the following two assumptions.

Assumption 11. The (extended-real-valued) functions and are proper, closed, and convex.

Assumption 12. The Lagrangian function has a saddle point; that is, there exists , not necessarily unique, for whichholds for all , , and .

The convergence of the ADMM for real convex separable optimization is established in the following theorem.

Theorem 13 (Section  3.2.1 in [2]). Under Assumptions 11 and 12, the ADMM iterates (28) satisfy the following.
(1) Residual Convergence. as ; that is, the iterates approach feasibility.
(2) Objective Convergence. as ; that is, the objective function of the iterates approaches the optimal value.
(3) Dual Variable Convergence. as , where is a dual optimal point.

3.2. ADMM for Complex Convex Separable Optimization

According to (10), we can consider the real functions in complex variables and . Then, the convex separable optimization of real functions in complex variables becomes where and are proper, closed, and convex functions; and are closed convex sets; and are given matrices; and is a given vector.

From (10) and Lemma 1, we can conclude that the complex convex separable optimization (30) is equivalent to the following convex separable optimization problem: The Lagrangian function of (31) is where . Then, the augmented Lagrangian function of (31) is where is called the penalty parameter. The ADMM for complex convex separable optimization is composed of the iterations Let . Then we havewhere is the scaled dual variable. Using the scaled dual variable, we can express the ADMM iterations (34) as

3.3. Optimality Conditions

The necessary and sufficient optimality conditions for the ADMM problem (31) are primal feasibility,and dual feasibility, Because minimizes , we have This means that and always satisfy (39); thus attaining optimality leads to satisfying (37) and (38).

Because minimizes , we have or equivalentlyFrom (38), can be viewed as a residual for the dual feasibility condition. By (37), can be viewed as a residual for the primal feasibility condition. These two residuals converge to zero as the ADMM proceeds.

3.4. Convergence

Similar to the ADMM for separable convex optimization in the real number domain, we can establish the convergence of the ADMM for complex separable convex optimization.

In this paper, we make the following two assumptions on the separable convex optimization of the real functions in complex variables.

Assumption 14. The (extended-real-valued) functions and are proper, closed, and convex.

Assumption 15. The Lagrangian function (32) has a saddle point; that is, there exists , not necessarily unique, for which holds for all , , and .

Theorem 16. Under Assumptions 14 and 15, the ADMM iterations (36) have the following conclusions.
(1) Residual Convergence. as ; that is, the iterates approach feasibility.
(2) Objective Convergence. as ; that is, the objective function of the iterates approaches the optimal value.
(3) Dual Variable Convergence. as , where is a dual optimal point.

Proof. Let be the saddle point for and . Then we have Since and is equivalent to , then we have From Theorem 2 and Examples 3 and 4, we get Note that is a real-valued function; then we getBy (36), minimizes for is convex, which is subdifferentiable, and so is . Based on Theorem 2, the optimality condition is Since we have which implies that minimizes Similarly, we may have that minimizes From (51) and (52), we have From (53) and , we can make the conclusion that where .
Adding (45) and (54), we get Let By following manipulation and rewriting of (55), we have Rewriting the first term of (55) as and then substituting into the first two terms in (58) give Since this can be expressed as Now let us regroup the remaining terms, that is, where is taken from (62). Substituting into the last term in (63) yields and substituting into the last two terms in (63), we get It implies that (55) can be expressed as To obtain (57), it suffices to show that the middle term of the expanded right-hand side of (68) is positive. To understand this, by reviving that minimizes and minimizes , we can add to obtain that Since , if we substitute we can get (57).
This means that decreases in each iteration by an amount depending on the norm of the residual and on the change in over one iteration. Since , it follows that and are bounded. Iterating the inequality above gives implying that and as . From (45), we have as . Furthermore, since as , we have as . This completes the proof.

3.5. Stopping Criterion

We can find that Substituting this into (54), we get This means that when the two residuals are small, the error must be small. Thus an appropriate termination criterion is that the primal residuals and dual residuals are small simultaneously; that is, and , where and are tolerances for the primal and dual feasibility, respectively.

4. Basis Pursuit with Complex ADMM

Consider the equality-constrained minimization problem in the complex number domain where is a given matrix, , and is a given vector.

Recall that Then In the form of the ADMM, the BP method can be expressed as where is the indicator function of ; that is, for and otherwise. Then, with the idea in [40], the ADMM iterations are provided as follows:The -update, which involves solving a linearly constrained minimum Euclidean norm problem, can be written as Let . Then, (82) is equivalent to where .

Lemma 17 (see [41]). The minimum-norm least-squares solution of is , where is the Moore-Penrose inverse of matrix .

Theorem 18. The -update of (81) is

Proof. As is of full row rank, its full-rank factorization is . Then it yields that [41] From Lemma 17, we have that is, Since is of full rank, we can rearrange (87) to obtain This completes the proof.

If problem (82) is in the real number domain, we have which is the same one obtained in Section [2].

The -update can be solved by the soft thresholding operator in the following theorem, which is a generalization of the soft thresholding in [2].

Theorem 19. Let . Then one has the following.
(1) If is real-valued, that is, , the soft thresholding operator is (2) If is purely imaginary, that is, , the soft thresholding operator is (3) If , the soft thresholding operator is where and

Proof. (1) Assume that , the updating of becomes minimizing the following function: From Theorem 2, we have This implies that . Then (93) can be rewritten as When , should be positive. Then, we haveIt is clear that this is a simple parabola, and its results areWhen , we can get the similar results as follows:From the above discussion, we can complete the proof.
(2) Assume that ; we have and by adopting the same approach in the above (1), we can get the results.
(3) Assume that and satisfy ; thenIt follows from Theorem 2 that By resolving it, we may get , , where and . Other cases can be discussed similarly. Thus we omit the proof here. This completes the proof.

From what has been discussed above on -update and -update, the iteration of the BP algorithm is where is projection onto and is the soft thresholding operator in the complex number domain.

5. Numerical Simulation

We give two numerical simulations with random data and EEG data. All our numerical experiments are carried out on a PC with Intel Core i7-4710MQ CPU at 2.50 GHz and 8 GB of physical memory. The PC runs MATLAB Version: R2013a on Window 7 Enterprise 64-bit operating system.

5.1. Numerical Simulation of the BP Algorithm with Random Data

Assume that is a discrete complex signal interested. itself is -sparse, which contains (at most) nonzero entries with . Select measurements uniformly at random matrix via . Hence reconstructing signal from measurement is generally an ill-posed problem which is an undetermined system of linear equations. However, the sparsest solution can be obtained by solving the constrained optimization problem: where is the -norm of . Unfortunately, (103) is a combinatorial optimization problem of which the computational complexity grows exponentially with the signal size . A key result in [17, 18] is that if is sparse, the sparsest solution of (103) can be obtained with overwhelming probability by solving the convex optimization problem (77). Next, we consider CS which is actually a kind of application of the BP method in complex variables.

5.1.1. The Effects of the Parameter

We demonstrate the complex signal sampling and recovery techniques with a discrete-time complex signal of length with sparsity which is generated randomly. is a random sensing matrix with . The variables and are initialized to be zero. We set the two tolerances of primal and dual residuals equal to . In order to understand the effects of the parameter on the convergence, we set the penalty parameter from 0.1 to 20 with the step 0.1. We have repeated the same experiment 100 times with the same parameter . The average runtime, the average numbers of iterations, and the average primal and dual errors of the ADMM for the different choices of the parameter are presented in Figure 1.

It is clear from Figure 1 (top) that when , the average runtime and the average iterations are reasonable. From Figure 1 (bottom), we can observe that, with the parameter increasing, the primal error decreases while the dual error becomes bigger. Numerical simulations suggest that choosing could accelerate the convergence of the ADMM.

5.1.2. The Effects of Tolerances for the Primal and Dual Residuals

Now, we take the different tolerances for the primal residuals and the dual residuals to analyze the performance of the ADMM, where sparse , measurements , the penalty parameter , and . We take two different signal lengths and . We have repeated the same experiment 100 times by a set of randomly generated data. For different choices of , the average numbers of iterations and the executing time of the above ADMM algorithm (102) are presented in Table 1. It is shown that, with the increasing of precision, the number of iterations increases accordingly while increasing of executing time is not obvious.

In Figure 2(a), the full line describes the changes of the primal residuals . In Figure 2(b), the full line describes the changes of the dual residuals . The dotted lines in Figures 2(a) and 2 (b) represent the original residual tolerance and dual residuals tolerance , respectively. From Figure 2, we can see that the two residuals descend monotonously.

5.2. Reconstruction of Electroencephalogram Signal by Using Complex ADMM

Electroencephalogram (EEG) signal is a weak bioelectricity of brain cells group, which can be recorded by placing the available electrodes on the scalp or intracranial detect. The EEG signal could reflect the brain bioelectricity rhythmic activity regularity of random nonstationary signal. In this area, much is known for clinical diagnosis and brain function [42]. Because EEG data is large, a very meaningful work is to compress EEG data. It is to effectively reduce the amount of data at the same time and to guarantee that the main features basically remain unchanged [43, 44].

In this paper, EEG signals are recorded with a g.USBamp and a g.EEGcap (Guger Technologies, Graz, Austria) with a sensitivity of 100 V, band pass filtered between 0.1 and 30 Hz, and sampled at 256 Hz. Data are recorded and analyzed using the ECUST BCI platform software package developed through East China University of Science and Technology [45, 46].

We get the complex signal by performing the discrete Fourier transform (DFT) on EEG signal ; that is, , with signal with length . Original EEG signal is not sparse, but its DFT signal becomes the approximate sparse signal; see Figure 3. The hard threshold of is properly set, which leads to with 90 percent zero valued entries, and is the approximation of .

To get the compression of sparse signal , we can first calculate from . is a random complex matrix of size , where , , and the sampling rate is 40%. Substituting and into the optimization model as presented in (77), that is, we can obtain the sparse optimal solution by employing the ADMM algorithm (102) in Section 4, in which is a good approximation of . By applying the inverse discrete Fourier transform (IDFT) on , we can get the approximation of original signal ; that is,The original signal and its reconstruction signal can be seen in Figure 4, in which (a) is the original signal , (b) is the reconstruction signal , and (c) is the comparison of them. With the comparison of (a), (b), and (c) in Figure 4, we can observe that the reconstructed signal is in good agreement with the original signal and retains the leading characteristic. The relative error .

Now we separate the complex signal into the real part and the imaginary part and then recast it into an equivalent real-valued optimization problem. We can calculate from and from , respectively. Here is a random real matrix of size , where , , and the sampling rate is 40% which is similar to the one used in the complex number domain. Substitute and into the optimization model as presented in (77); that is, Although and can approach and , respectively, the reconstructed signal is not consistent with the original signal ; see Figure 5. The relative error . It can be seen that our new ADMM proposed in this paper performs better than the classic ADMM.

6. Conclusions

In this paper, the ADMM for separable convex optimization of real functions in complex variables has been studied. By using Wirtinger calculus, we have established the convergence of the algorithm, which is the generalization of the one obtained in real variables. Furthermore, the BP algorithm is given in the form of the ADMM, in which projection algorithm and the soft thresholding formula are generalized from the real number domain to the complex case. The simulation results demonstrate that the ADMM can quickly solve convex optimization problems in complex variables within the scopes of the signal compression and reconstruction, which is better than the results in the real number domain.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors would like to thank Dr. Shouwei Zhao and Dr. Zhongtuan Zheng for pointing out some typos and inconsistencies in an earlier version of the paper. This research was supported by Shanghai Natural Science Fund Project (no. 14ZR1418900), the National Natural Science Foundation of China (no. 11471211), and the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry.