Research Article | Open Access

# A Regularization SAA Scheme for a Stochastic Mathematical Program with Complementarity Constraints

**Academic Editor:**Song Cen

#### Abstract

To reflect uncertain data in practical problems, stochastic versions of the mathematical program with complementarity constraints (MPCC) have drawn much attention in the recent literature. Our concern is the detailed analysis of convergence properties of a regularization sample average approximation (SAA) method for solving a stochastic mathematical program with complementarity constraints (SMPCC). The analysis of this regularization method is carried out in three steps: First, the almost sure convergence of optimal solutions of the regularized SAA problem to that of the true problem is established by the notion of epiconvergence in variational analysis. Second, under MPCC-MFCQ, which is weaker than MPCC-LICQ, we show that any accumulation point of Karash-Kuhn-Tucker points of the regularized SAA problem is almost surely a kind of stationary point of SMPCC as the sample size tends to infinity. Finally, some numerical results are reported to show the efficiency of the method proposed.

#### 1. Introduction

Our concern in this paper is the following stochastic mathematical program with complementarity constraints (SMPCC): where , , , , and are random mappings; is a random vector defined on a probability space ; denotes the mathematical expectation; the notation means “perpendicular.” Throughout the paper, we assume that , , , , and are all well defined and finite for any . To ease the notation, we write as and this should be distinguished from being a deterministic vector of in a context.

The SMPCC (1) is a natural extension of deterministic mathematical program with complementarity constraints (MPCC) [1, 2], which have many applications in transportation [3] and communication networks [4], and so forth. There are many stochastic formulations of MPCC proposed in the recent years [3, 5–7]. Among these formulations, Birbil et al. [3] applied sample path method [8] to SMPCC (1).

In this paper, we are concerned with a numerical method for solving (1). Evidently, if the integral involved in the mathematical expectation of problem (1) can be evaluated either analytically or numerically, then problem (1) can be regarded as the usual MPCC problem and consequently it can be solved by existing numerical methods that are related. However, as shown in [9], in many situations, exact evaluation of the expected value in (1) for is either impossible or prohibitively expensive. Sample average approximation (SAA) method [8, 10] is suggested by many authors to handle such difficulty; see the recent works [11–15]. The basic idea of SAA is to generate an independent identically distributed (iid) sample of and then approximate the expected value with sample average. In this context, let be iid sample; then the SMPCC (1) is approximated by the following SAA problem: where , , , , is the sample-average function of , , , and respectively. We refer to (1) as the true problem and (2) as the SAA problem to (1). Another critical problem for solving (1) is how to solve SAA problem (2) effectively. Since the Mangasarian-Fromovitz constraint qualification is violated at every feasible point of SAA problem (2) (see [16]), it is not appropriate to use standard nonlinear programming software to solve the SAA problem directly. The well-known regularization scheme [17], is a effective way to deal with this issue. That is, by replacing the complementarity constraint with a parameterized system of inequalities, the SAA problem is reformulated as follows: where is a parameter, “” denotes the Hadamard product and is a vector with components 1. Then the SAA problem can be approximated by a smooth nonlinear programming (NLP) problem (3) when the parameter is sufficiently small. Consequently, a solution to true problem (1) can be obtained by solving a sequence of such regularized SAA problems.

In this paper, we focus on the detailed analysis of convergence properties of the regularized SAA problem (3) to the true problem (1) as the sample size tends to infinity. The main contributions of this paper can be summarized as follows: by the notion of epiconvergence in [18], we establish the almost sure convergence of optimal solutions of smoothed SAA problem as the sample size tends to infinity. Under MPCC-MFCQ, we show that any accumulation point of Karash-Kuhn-Tucker points of the regularized SAA problem is a kind of stationary point almost surely. The obtained results can be seen an improvement of [17, Theorem 3.1] for solving SMPCC under weaker constraint qualification conditions. Moreover, under the MPCC strong second-order sufficient condition (MPCC-SSOSC) in [16], we investigate sufficient conditions under which the smoothed SAA problem possesses a Karash-Kuhn-Tucker point when the sample size is large enough, and the sequence of those points converges exponentially to a kind of stationary point of SMPCC almost surely as the sample size tends to infinity.

This paper is organized as follows: Section 2 gives preliminaries needed throughout the whole paper. In Sections 3 and 4, we establish the almost sure convergence of optimal solutions and stationary points of the regularized SAA problem as the sample size tends to infinity respectively. In Section 5, existence and exponential convergence rate of stationary points of the regularized SAA problem are investigated. We also report some preliminary numerical results in Section 6.

#### 2. Preliminaries

Throughout this paper we use the following notations. Let denote the Euclidean norm of a vector or the Frobenius norm of a matrix. For a matrix , denotes the element of the th row and th column of . We use to denote the identity matrix, denotes the closed unite ball, and denotes the closed ball around of radius . For a extended real-valued function , , , and denote their epigraph that is, the set , the gradient of at , and the Hessian matrix of at , respectively. For a mapping , denotes the Jacobian of at . stands for the positive real numbers.

In the following, we introduce some concepts of the convergence of set sequences and mapping sequences in [18] which will be used in the next section. Define where denotes the set of all positive integer numbers.

*Definition 1. *For sets and in with closed, the sequence is said to converge to (written ) if
with

The continuous properties of a set-valued mapping can be developed by the convergence of sets.

*Definition 2. *A set-valued mapping is continuous at , symbolized by , if

*Definition 3. *Consider now a family of functions , where . One says that epiconverges to a function as and is written as
if the sequence of sets epi converges to epi in as .

*Definition 4. *Given a clos set and a point . The cone
is called the Fréchet normal cone to at . Then the limiting normal cone (also known as Mordukhovich normal cone or basic normal cone) to at is defined by
If is a closed convex set, the limiting normal cone is the normal cone in the sense of convex analysis.

Next, we recall some basic concepts that are often employed in the literature on optimization problems with complementarity constraints.

Let be a feasible point of problem (1) and for convenience we define the index sets

The constraint qualifications for SMPCC is as follows.

*Definition 5. *Assume , , , and are continuously differentiable at . We say the MPCC Mangasarian-Fromovitz constraint qualification (MPCC-MFCQ) holds at if the set of vectors
are linearly independent and there exists a nonzero vector such that

*Definition 6. *Assume , , , and are continuously differentiable at . We say the MPCC linear independence constraint qualification (MPCC-LICQ) holds at if the set of vectors
are linearly independent.

As in [16], we use the following two stationarity concepts for SMPCC.

*Definition 7. *Assume is a feasible point of SMPCC (1), , , , and are continuously differentiable at . Suppose there exist vectors , , , and such that satisfies the following conditions:
(i)(-stationary point) We call a Clarke stationary point of (1) if , .(ii)(-stationary point) We call a strongly stationary point of (1) if , , .

The following upper level strict complementarity condition was used in [16] in the context of sensitivity analysis for MPCC.

*Definition 8. *We say that the upper level strict complementarity condition (ULSC) holds at if and , the multipliers correspondence to , and , respectively, satisfy for all .

It is well known that a point satisfies the lower level strict complementarity condition (LLSC) if hold for all , we can see from an example from [16] that ULSC condition is considerably weaker than the LLSC condition, and in practice, it may make more sense than the latter one.

We use the following second-order condition based on the MPCC-Lagrangian: of ().

*Definition 9 (see [16]). *Let be a -stationary point of (1) and is the corresponding multiplier at . Suppose , , , , and are twice continuously differentiable at . We say that the MPCC strong second-order sufficient condition (MPCC-SSOSC) holds at if
for every nonvanishing with
Assume is a -stationary point of (1) and is the corresponding multiplier. Then we know from [16, Theorem 7] that if MPCC-SSOSC holds at , it is a strict local minimizer of the SMPCC (1).

Throughout the paper, we assume the sample of the random vector is iid and give the following assumptions to make (1) more clearly defined and to facilitate the analysis.

*Assumption 10. *The mapping , , , , and are twice continuously differentiable on a.e. .

*Assumption 11. *For any , there exists a closed bounded neighborhood of and a nonnegative measurable function such that and
for all , where is any element in the collection of functions , , , , , , , , , and .

*Assumption 12. *For every , the following properties hold ture.(A1)For every , the moment generating function
of random variable is finite valued for all in a neighborhood of zero.(A2)There exists a measurable function such that
for all and .(A3)The moment generating of is finite valued for all in a neighborhood of zero.

Assumptions 10–12 are popularly used conditions for the analysis of SAA method for stochastic programming. Under Assumptions 10−11, we know from [10, Chapter 7], that and are twice continuously differentiable on . In particular,
Assumption 12 is used to ensure exponential convergence rate of proposed regularization SAA method in Section 5.

The following results are directly from the Uniform Laws of Large Numbers in [10, Theorem 7.48].

Lemma 13. *Let be a feasible point of (1). Suppose that Assumptions 10−11 are satisfied; then we obtain
**
where the set is a closed bounded neighborhood of and is any element in the collection of functions , , , , , , , , , and .*

#### 3. Almost Sure Convergence of Optimal Solutions

In this section, by the notion of epiconvergence in [18], we establish the almost convergence of optimal solutions of regularized SAA problem (3) to those of SMPCC (1) as the sample size tends to infinity.

Let us introduce some notions:

Now we give a conclusion about the almost sure convergence of the set as tends to infinity in the following proposition.

Proposition 14. *Let as . Suppose Assumptions 10−11 hold. If MPCC-LICQ (Definition 6) holds for any , then
*

*Proof. *We at first show that w.p.1. It suffices to prove that for a sequence satisfying for each , if converges to w.p.1 as , then w.p.1. Indeed, we know from the definition of that satisfies
for ; and , which, by Lemma 13, means that w.p.1.

Let . Next we show that w.p.1. Let
where the mapping is defined by
Then , where
Under MPCC-LICQ, has Aubin property [18] around , which means that there exist constants , and such that
holds for and . Therefore, for sufficiently small positive numbers , there exists a continuous function such that and for any ,
Let
Then, by Lemma 13, we have for large enough
and for any ,
Define a function
This is a continuous mapping from the compact convex set to itself. By Brouwer's fixed theorem, has a fixed point. Hence, there exists a vector w.p.1 such that . Therefore, we have from (31) that
That is, , where
By Lemma 13, we obtain for sufficiently large , due to , where
which means that . As a result, belongs to w.p.1 because of the almost sure convergence of to as . We complete the proof.

By Definition 3, similarly to the proof of [15, Lemma 4.3], we obtain the following lemma.

Lemma 15. *Under the conditions of Proposition 14, we have
*

The following result is directly from [18, Theorem 7.31].

Theorem 16. *Suppose solves (3) for each and is almost surely an accumulate point of the sequence . If the conditions in Proposition 14 hold and is finite, then is almost surely an optimal solution of the true problem (1).*

#### 4. Almost Sure Convergence of Stationary Points

In practice, finding a global minimizer might be difficult and in some cases we might just find a stationary point. As a result, we want to know whether or not an accumulation point of the sequence of stationary points of regularized SAA problem (3) is almost surely a kind of stationary point of SMPCC (1).

Notice that (3) is a standard nonlinear programming with smooth constraints. If is a local optimal solution of the regularized SAA problem (3), then under some constraint qualifications, is a stationary point of (3); namely, there exists Lagrange multipliers , , , , and such that the vector satisfies the following Karash-Kuhn-Tucker (KKT) condition for problem (3): with

We now prove the almost sure convergence of the regularization SAA method for SMPCC (1).

Theorem 17. *Suppose Assumptions 10−11 hold. Let and let be a stationary point of problem (3). If the sequence converges to w.p.1 as and MPCC-MFCQ (Definition 5) holds at , then the following statements hold:*(i)* is a -stationary point of SMPCC (1) almost surely.*(ii)*If, in addition, the multipliers and for all , where
then is a -stationary point of SMPCC (1) almost surely.*

*Proof. *Since is a stationary point of problem (3), there exist multipliers , , , , and such that
with
Then (43) can be reformulated as
with
Next we show that is almost surely bounded under the MPCC-MFCQ. We assume by contradiction that is unbounded, then there exists a number sequence such that , where
Since
and by outer semicontinuousness of normal cone
Notice that , , and , ; then by multiplying to both sides of (45) and taking limit, we have
with , where
However, we know from MPCC-MFCQ that for any and
which is called the generalized Robinson constraint qualification in [19]. Notice that for and , there exists such that
Then by dual form of generalized Robinson constraint qualification in Yen [19], we have for any
which means that
That is, in (50) is . This contradicts the condition that and hence is bounded. Without loss of generality, we assume w.p.1 as , where
Notice that
Then we know from (46) that for , in the case when due to and for each . In the case when , since
we have . As a result, by Definition 7, is a -stationary point. If for , then we know from Definition 7 that is a -stationary point. The proof is completed.

*Remark 18. *For a deterministic MPCC problem, Scholtes [17] studied the properties of the limit point of a sequence of stationary points generated by the same regularization method under MPCC-LICQ. Notice that MPCC-MFCQ in Theorem 17 is weaker than MPCC-LICQ. Thus this theorem can be seen as an improvement of [17, Theorem 3.1] for solving SMPCC under weaker constraint qualification conditions.

#### 5. Existence and Exponential Convergence Rate

In this section, we discuss the conditions ensuring existence and exponential convergence of stationary points of regularized SAA problem satisfying (40) when the sample size is sufficiently large.

We need the following lemma.

Lemma 19. *Let be a compact set. Suppose Assumptions 10–12 hold. Then for any , there exist positive constants and , independent of , such that
*

*Proof. *Under Assumptions 10–12, we know from [10, Theorem 7.65] that for each , there exist positive constants and , independent of , such that
where and denote the th component of and , respectively. Therefore, we have
where , and .

We now state our existence and exponential convergence results. The proof relies on an application of Robinson’s standard NLP stability theory in [20].

Theorem 20. *Let be a -stationary point of SMPCC (1) and . Suppose*(i)*Assumptions 10–12 hold at ,*(ii)*MPCC-LICQ (Definition 6), MPCC-SSOSC (Definition 9), and ULSC (Definition 8) hold at .**Then we have that*(a)*there exits satisfying stationary condition (40) of (3) w.p.1 for each when is sufficiently large and w.p.1 as N→∞;*(b)*the sequence in (a) satisfies that for every , there exist positive constants and , independent of , such that
for sufficiently large.*

*Proof. *Since is a -stationary point of SMPCC, then there exist vectors , , , and such that
where
with
Notice that (63) can be seen as a KKT condition of the following NLP problem:
The MPCC-SSOSC ensures the strong second-order sufficient condition for NLP problem (67), which, under MPCC-LICQ, implies the stability of (67) in the sense of Robinson [20]. Hence, there exist positive numbers , and such that for every , the mapping has only one solution with and the mapping satisfying

Since ULSC holds at and is a -stationary point, we have for , which means that for sufficiently small and any , , and . Let
where