Journal of Applied Mathematics

Volume 2012 (2012), Article ID 410137, 10 pages

http://dx.doi.org/10.1155/2012/410137

## Strong Convergence of a Projected Gradient Method

Department of Mathematics, Tianjin Polytechnic University, Tianjin 300387, China

Received 16 January 2012; Accepted 5 February 2012

Academic Editor: Yeong-Cheng Liou

Copyright © 2012 Shunhou Fan and Yonghong Yao. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

The projected-gradient method is a powerful tool for solving constrained convex optimization problems and has extensively been studied. In the present paper, a projected-gradient method is presented for solving the minimization problem, and the strong convergence analysis of the suggested gradient projection method is given.

#### 1. Introduction

In the present paper, our main purpose is to solve the following minimization problem: where is a nonempty closed and convex subset of a real Hilbert space , is a real-valued convex function.

Now it is well known that the projected-gradient method is a powerful tool for solving the above minimization problem and has extensively been studied. See, for instance, [1–8]. The classic algorithm is the following form of the projected-gradient method: where is an any constant, is the nearest point projection from onto , and denotes the gradient of .

It is known [1] that if has a Lipschitz continuous and strongly monotone gradient, then the sequence generated by (1.2) can be strongly convergent to a minimizer of in . If the gradient of is only assumed to be Lipschitz continuous, then can only be weakly convergent if is infinite dimensional. An interesting problem is how to appropriately modify the projected gradient algorithm so as to have strong convergence? For this purpose, recently, Xu [9] introduced the following algorithm: Under some additional assumptions, Xu [9] proved that the sequence converges strongly to a minimizer of (1.1). At the same time, Xu [9] also suggested a regularized method: Consequently, Yao et al. [10] proved the strong convergence of the regularized method (1.4) under some weaker conditions.

Motivated by the above works, in this paper we will further construct a new projected gradient method for solving the minimization problem (1.1). It should be pointed out that our method also has strong convergence under some mild conditions.

#### 2. Preliminaries

Let be a nonempty closed convex subset of a real Hilbert space . A bounded linear operator is said to be strongly positive on if there exists a constant such that A mapping is called nonexpansive if A mapping is said to be an averaged mapping, if and only if it can be written as the average of the identity and a nonexpansive mapping; that is, where is a constant and is a nonexpansive mappings. In this case, we call is -averaged.

A mapping is said to be -inverse strongly monotone (-ism), if and only if The following proposition is well known, which is useful for the next section.

Proposition 2.1 (See [9]). *(1) The composite of finitely many averaged mappings is averaged. That is, if each of the mappings is averaged, then so is the composite . In particular, if is -averaged and is -averaged, where , then the composite is -averaged, where .**(2) T is -ism, then for , is ()-ism.*

Recall that the (nearest point or metric) projection from onto , denoted by , assigns, to each , the unique point with the property We use to denote the solution set of (1.1). Assume that (1.1) is consistent, that is, . If is Frechet differentiable, then solves (1.1) if and only if satisfies the following optimality condition: where denotes the gradient of . Observe that (2.6) can be rewritten as the following VI (Note that the VI has been extensively studied in the literature, see, for instance [11–25].) This shows that the minimization (1.1) is equivalent to the fixed point problem where is an any constant. This relationship is very important for constructing our method.

Next we adopt the following notation:(i) means that converges strongly to ;(ii) means that converges weakly to ;(iii) is the fixed points set of .

Lemma 2.2 (See [26]). *Let and be bounded sequences in a Banach space and let be a sequence in with
**
Suppose
**
for all and
**
Then, .*

Lemma 2.3 (See [27] (demiclosedness principle)). * Let be a closed and convex subset of a Hilbert space and let be a nonexpansive mapping with . If is a sequence in weakly converging to and if converges strongly to , then
**
In particular, if , then .*

Lemma 2.4 (See [28]). *Assume is a sequence of nonnegative real numbers such that
**
where is a sequence in and is a sequence such that*(1)*;*(2)* or .**Then, .*

#### 3. Main Results

Let be a closed convex subset of a real Hilbert space . Let be a real-valued Frechet differentiable convex function with the gradient . Let be a -contraction. Let be a self-adjoint, strongly positive bounded linear operator with coefficient . First, we present our algorithm for solving (1.1). Throughout, we assume .

*Algorithm 3.1. *For given , compute the sequence iteratively by
where are two constants and the real number sequence .

*Remark 3.2. *In (3.1), we use two projections. Now, it is well-known that the advantage of projections, which makes them successful in real-word applications, is computational.

Next, we show the convergence analysis of this Algorithm 3.1.

Theorem 3.3. *Assume that the gradient is -Lipschitzian and . Let be a sequence generated by (3.1), where is a constant and the sequence satisfies the conditions: (i) and (ii) . Then converges to a minimizer of (1.1) which solves the following variational inequality:
*

By Algorithm 3.1 involved in the projection, we will use the properties of the metric projection for proving Theorem 3.3. For convenience, we list the properties of the projection as follows.

Proposition 3.4. *It is well known that the metric projection of onto has the following basic properties:*(i)*, for all ;*(ii)*, for every ;*(iii)*, for all , .*

*The Proof of Theorem 3.3*

Let . First, from (2.8), we note that . By (3.1), we have
Thus, by induction, we obtain

Note that the Lipschitz condition implies that the gradient is -inverse strongly monotone (ism), which then implies that is -ism. So, is -averaged. Now since the projection is -averaged, we see that is -averaged. Hence we have that
where are nonexpansive and . Then we can rewrite (3.1) as
where
Set for all . Since is bounded, we deduce , , and are all bounded. Hence, there exists a constant such that
Thus,
It follows that
This together with Lemma 2.2 implies that
So,
Since
we deduce
Next we prove
where is the unique solution of VI (3.2).

Indeed, we can choose a subsequence of such that
Since is bounded, there exists a subsequence of which converges weakly to a point . Without loss of generality, we may assume that converges weakly to . Since , is nonexpansive. Thus, from (3.14) and Lemma 2.3, we have . Therefore,
Finally, we show . By using the property of the projection , we have
It follows that
It is obvious that . Then we can apply Lemma 2.4 to the last inequality to conclude that . The proof is completed.

In (3.1), if we take and , then (3.1) reduces to the following.

*Algorithm 3.5. *For given , compute the sequence iteratively by
where are two constants and the real number sequence .

From Theorem 3.3, we have the following result.

Theorem 3.6. *Assume that the gradient is -Lipschitzian and . Let be a sequence generated by (3.20), where is a constant and the sequences satisfies the conditions: (i) and (ii) . Then converges to a minimizer of (1.1) which is the minimum norm element in .*

*Proof. *As a consequence of Theorem 3.3, we obtain that the sequence generated by (3.20) converges strongly to which satisfies
This implies
Thus,
That is, is the minimum norm element in . This completes the proof.

#### Acknowledgment

Y. Yao was supported in part by Colleges and Universities Science and Technology Development Foundation (20091003) of Tianjin, NSFC 11071279 and NSFC 71161001-G0105.

#### References

- E. S. Levitin and B. T. Polyak, “Constrained minimization problems,”
*USSR Computationnal Mathematics and Mathematical Phsics*, vol. 6, pp. 1–50, 1966. View at Google Scholar - E. M. Gafni and D. P. Bertsekas, “Two metric projection methods for constrained optimization,”
*SIAM Journal on Control and Optimization*, vol. 22, no. 6, pp. 936–964, 1984. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - P. H. Calamai and J. J. More, “Projected gradient methods for linearly constrained problems,”
*Mathematical Programming*, vol. 39, no. 1, pp. 93–116, 1987. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - B. T. Polyak,
*Introduction to Optimization*, Translations Series in Mathematics and Engineering, Optimization Software, New York, NY, USA, 1987. - C. Wang and N. Xiu, “Convergence of the gradient projection method for generalized convex minimization,”
*Computational Optimization and Applications*, vol. 16, no. 2, pp. 111–120, 2000. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - N. H. Xiu, C. Y. Wang, and J. Z. Zhang, “Convergence properties of projection and contraction methods for variational inequality problems,”
*Applied Mathematics and Optimization*, vol. 43, no. 2, pp. 147–168, 2001. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - A. Ruszczynski,
*Nonlinear Optimization*, Princeton University Press, Princeton, NJ, USA, 2006. - N. H. Xiu, C. Y. Wang, and L. Kong, “A note on the gradient projection method with exact stepsize rule,”
*Journal of Computational Mathematics*, vol. 25, no. 2, pp. 221–230, 2007. View at Google Scholar · View at Zentralblatt MATH - H. K. Xu, “Averaged mappings and the gradient-projection algorithm,”
*Journal of Optimization Theory and Applications*, vol. 150, no. 2, pp. 360–378, 2011. View at Publisher · View at Google Scholar - Y. Yao, S. M. Kang, J. Wu, and P. X. Yang, “A regularized gradient projection method for the minimization problem,”
*Journal of Applied Mathematics*, vol. 2012, Article ID 259813, 9 pages, 2012. View at Publisher · View at Google Scholar - Y. Yao, M. A. Noor, and Y. C. Liou, “Strong convergence of a modified extra-gradient method to the minimum-norm solution of variational inequalities,”
*Abstract and Applied Analysis*, vol. 2012, Article ID 817436, 9 pages, 2012. View at Publisher · View at Google Scholar - Y. Yao, M. A. Noor, Y. C. Liou, and S. M. Kang, “Iterative algorithms for general multi-valued variational inequalities,”
*Abstract and Applied Analysis*, vol. 2012, Article ID 768272, 10 pages, 2012. View at Publisher · View at Google Scholar - Y. Yao, Y. C. Liou, and S. M. Kang, “Two-step projection methods for a system of variational inequalityproblems in Banach spaces,”
*Journal of Global Optimization*. In press. View at Publisher · View at Google Scholar - M. A. Noor, “Some developments in general variational inequalities,”
*Applied Mathematics and Computation*, vol. 152, no. 1, pp. 199–277, 2004. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - G. M. Korpelevich, “An extragradient method for finding saddle points and for other problems,”
*Ekonomika i Matematicheskie Metody*, vol. 12, no. 4, pp. 747–756, 1976. View at Google Scholar - L. C. Ceng and J. C. Yao, “An extragradient-like approximation method for variational inequality problems and fixed point problems,”
*Applied Mathematics and Computation*, vol. 190, no. 1, pp. 205–215, 2007. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - Y. Yao and J. C. Yao, “On modified iterative method for nonexpansive mappings and monotone mappings,”
*Applied Mathematics and Computation*, vol. 186, no. 2, pp. 1551–1558, 2007. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - B. S. He, Z. H. Yang, and X. M. Yuan, “An approximate proximal-extragradient type method for monotone variational inequalities,”
*Journal of Mathematical Analysis and Applications*, vol. 300, no. 2, pp. 362–374, 2004. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - Y. Yao, R. Chen, and H. K. Xu, “Schemes for finding minimum-norm solutions of variational inequalities,”
*Nonlinear Analysis*, vol. 72, no. 7-8, pp. 3447–3456, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - Y. Yao and M. A. Noor, “On viscosity iterative methods for variational inequalities,”
*Journal of Mathematical Analysis and Applications*, vol. 325, no. 2, pp. 776–787, 2007. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - Y. Yao and M. A. Noor, “On modified hybrid steepest-descent methods for general variational inequalities,”
*Journal of Mathematical Analysis and Applications*, vol. 334, no. 2, pp. 1276–1289, 2007. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - Y. Yao, M. A. Noor, K. I. Noor, Y. C. Liou, and H. Yaqoob, “Modified extragradient methods for a system of variational inequalities in Banach spaces,”
*Acta Applicandae Mathematicae*, vol. 110, no. 3, pp. 1211–1224, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - L. C. Ceng, N. Hadjisavvas, and N. C. Wong, “Strong convergence theorem by a hybrid extragradient-like approximation method for variational inequalities and fixed point problems,”
*Journal of Global Optimization*, vol. 46, no. 4, pp. 635–646, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - F. Cianciaruso, V. Colao, L. Muglia, and H. K. Xu, “On an implicit hierarchical fixed point approach to variational inequalities,”
*Bulletin of the Australian Mathematical Society*, vol. 80, no. 1, pp. 117–124, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - X. Lu, H. K. Xu, and X. Yin, “Hybrid methods for a class of monotone variational inequalities,”
*Nonlinear Analysis*, vol. 71, no. 3-4, pp. 1032–1041, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - T. Suzuki, “Strong convergence theorems for infinite families of nonexpansive mappings in general Banach spaces,”
*Fixed Point Theory and Applications*, no. 1, pp. 103–123, 2005. View at Google Scholar · View at Zentralblatt MATH - K. Goebel and W. A. Kirk,
*Topics in Metric Fixed Point Theory*, vol. 28 of*Cambridge Studies in Advanced Mathematics*, Cambridge University Press, Cambridge, UK, 1990. View at Publisher · View at Google Scholar - H. K. Xu, “Iterative algorithms for nonlinear operators,”
*Journal of the London Mathematical Society*, vol. 66, no. 1, pp. 240–256, 2002. View at Publisher · View at Google Scholar · View at Zentralblatt MATH