#### Abstract

We obtain a new Taylor's formula in terms of the order subdifferential of a function from to . As its applications in optimization problems, we build order sufficient optimality conditions of this kind of functions and order necessary conditions for strongly -quasiconvex functions.

#### 1. Introduction

For a function from to , Luc [1] studied the order subdifferential of it, established a Taylor-type formula in terms of such order subdifferential, and applied such Taylor-type formula to consider two-order optimality conditions in vector optimization and characterizations of quasiconvex functions. In vector optimization, notions of Pareto solution, weak Pareto solution, sharp minima and weak sharp minima are very important; see [2–14] and the references therein. Some authors have attained many necessary or sufficient optimality conditions in optimization problems. In particular, Zheng and Yang provided some results on sharp minima, and weak sharp minima for high-order smooth vector optimization problems in Banach spaces. By the tools of nonsmooth analysis, many optimality conditions were obtained; for examples, one can see [6, 7, 15, 16] and the references therein. Such optimality conditions play a key role in many issues of mathematical programming such as sensitivity analysis and error bounds.

Motivated by Luc [1] and Zheng and Yang [17], in this paper, we consider the order subdifferential and optimality conditions of a vector-valued function from to . We will first prove a new Taylor's formula in the terms of order subdifferential for functions from to , which is analogous to that for real-valued functions in [1]. Then, under the positive definiteness assumption of order subdifferential, we will use this formula to derive order optimality conditions of weak Pareto and Pareto solutions in the terms of order subdifferential for a function from to . Finally, we will define a kind of strongly -quasiconvex functions and prove a necessary condition in the terms of (th order subdifferential for such kind of functions. Our results extend the corresponding results in [1] for functions from to to that for vector-valued functions from to and in [17] for functions in smooth setting to that in nonsmooth setting, respectively.

The outline of the paper is as follows. In the next section, we give some notions and preliminary results in vector optimization problems. In Section 3, we build our Taylor's formula in the terms of order subdifferential for a function from to . In Section 4, as applications in optimization problems, we establish some optimality conditions in terms of th order subdifferential. In Section 5, we give a necessary condition in the terms of th order subdifferential for a strongly -quasiconvex vector-valued function.

#### 2. Preliminaries

Let be Banach spaces, the dual space of , a closed convex cone with , and the dual cone of ; that is, For , we define and if and , respectively. Let be a subset of and . Recall that (i) is a weak Pareto point of if there exists no point such that ; (ii) is a Pareto point of if there exists no point such that ; (iii) is an ideal point of if for all . Let , , and denote the sets of all weak Pareto, Pareto, and ideal points of , respectively. It is easy to verify that

Let be equipped with the norm .

Let be -linear and symmetric mapping [17]; that is, for any and , where is an arbitrary permutation of . Let be a mapping. It is known that its derivative is -linear, symmetric, and continuous mapping if is -time smooth.

Let be a function from to and be a closed convex cone. Consider the following vector optimization problem A vector is said to be a local weak Pareto (resp., Pareto and ideal) solution of (4) if there exists such that is a weak Pareto (resp., Pareto and ideal) point of , where denotes the open ball with center and radius . We say that is a sharp Pareto solution of (4) of order if there exist such that where .

We denote by , the class of -time differentiable mappings from to whose th order derivatives are locally Lipschitz mappings and by the class of locally Lipschitz functions from to . By Rademacher's theorem (see [18]), for any , , its th order derivative is a function differentiable almost everywhere. The th order subdifferential of at is defined as “generalized Jacobian” of at in Clarke's sense [18] as follows: It is worth mentioning that each element in is a linear and symmetric mapping from to . For more details about , we refer the reader to [18].

It is similar to the proof of Lemma 2.1 in [1], and one can verify the following chain rule.

Lemma 1. *Let in , be a function from to defined by for every , and let be a function from to . Then,
*

#### 3. A New Taylor's Formula in Form of High-Order Subdifferential

By Lemma 1, we have the following Taylor-type formula for a vector-valued function from to which will be useful in the sequel.

Theorem 2. *Let , , and be as in Lemma 1. Then, there exists such that
**
where denotes and
*

*Proof. *Let be a vector satisfying
We only need to show that there exists such that
Let be as Lemma 1. Set and
Let be arbitrarily given. Since the function is locally Lipschitz and , applying Lebourg mean value theorem [18, Theorem 2.3.7 and Theorem 2.3.9], there exists such that
Noting that and each have derivatives which are continuous, it follows that they are strictly -differentiable. We have
Here, the first equation holds by Propositions 7.4.3(b), and 7.3.5 in [19] and the second holds by Proposition 7.3.9 in [19]. By the chain rule [19, Theorem 7.4.5(a)], we also have
Hence, we have
From (13) and (16), we have
Together with Lemma 1, it follows that
that is,
Since is arbitrary in and is convex and compact, by the separation theorem, we can easily show that . Hence, we can take such that . The proof is completed.

Corollary 3. *Let be as in Theorem 2 and . Then, for every , there exist and a -linear mapping from to such that
*

*Proof. *By Theorem 2, for a given , there exists such that
Let be an element minimizing the distance from to the convex and compact set . Set
Then, from (21), we obtain the formula of the corollary. Moreover, since the mapping is upper continuous, nonempty, convex, and compact valued (see [18]), for any , there exists such that, for all (where denotes the closed unit ball of ),
where denotes the closed unit ball of the space of all bounded linear operators from to . If , then
With this we obtain . The proof is completed.

#### 4. The Positive Definiteness of High-Order Subdifferential and Optimality Conditions

Recall [17] that -linear symmetric mapping is said to be positively definite (resp., positively semidefinite) with respect to the ordering cone if where denotes . If is odd and the ordering cone is pointed (i.e., ), then is positively semidefinite if and only if ; see [17].

By the separation theorem, it is easy to verify that a -linear symmetric mapping is positively semidefinite with respect to the ordering cone if and only if the composite is positively semidefinite for any . Recall that a mapping is -convex if Noting that is -convex if and only if is convex for all , one can see that a twice differentiable function is -convex if and only if is positively semidefinite for all .

Inspired by the notion of positive definiteness, we introduce positive definiteness of the th order subdifferential for functions.

*Definition 4. *Let be a function from to and a closed convex cone of . We say that the th order subdifferential mapping is positively definite at with respect to the ordering cone if each is positively definite with respect to .

Proposition 5. *Let be a function from to , and let be a closed convex cone of . Suppose that the subdifferential mapping is positively definite at with respect to . Then, there exists such that
**
where . *

*Proof. *From [17, Proposition 3.4], for any , there exists such that
If the conclusion is not true, then, for every natural number , there exist , and such that
Since and are compact, we can assume that →, → (passing to a subsequence if necessary). Then,
for all . But from (28), for large enough , we have
which is a contradiction with (29). The proof is completed.

Under the positive definiteness assumption, we will provide a th order sufficient condition for to be a sharp local Pareto solution of (4) for a function .

Theorem 6. *Let be a function from to , a closed convex cone of , and . Suppose that there exists with such that , and that is positively definite at with respect to the ordering cone . Then, is a local Pareto solution of (4), and there exist such that
*

*Proof. *Since is positively definite with respect to , by Proposition 5, there exists such that
Noting that and , we have that
Let for all . Since is a function, so is . Noting that with Corollary 3, there exist and -linear mapping with such that
It follows that there exists such that
for all . Since , it follows from (34) and (36) that
On the other hand, for any , one has
This implies that (32) holds. It remains to show that is a local Pareto solution of (4). Let such that . Then, . It follows from (32) that , and hence . This shows that is a local Pareto solution of (4).

In Theorem 6, if is a -convex function, then is a global Pareto solution of (4).

Theorem 7. *Let be a -convex function from to , a closed convex cone of , and . Suppose that there exists with such that and that is positively definite. Then, is a global Pareto solution of (4), and there exists such that
*

*Proof. *Similar to the proof of Theorem 6, one can show that (39) implies that is a global Pareto solution of (4). It remains to show that (39) holds. By Theorem 6, there exist such that (32) holds. Since is -convex, it is easy to verify that is a convex function. Let . Then,
Hence, . Letting , it follows from (32) that (39) holds. The proof is completed.

With = in Theorem 7 replaced by a stronger assumption, we have the following sufficient condition for sharp ideal solutions of (4).

Theorem 8. *Let be a function from to , a closed convex cone of , and . Suppose that and that is positively definite at with respect to the ordering cone . Then, there exist such that
*

*Proof. *By Theorem 6, we need only to show that there exists such that (41) holds. Since is positively definite, there exists such that
It follows that
On the other hand, since , with Corollary 3, we can assume that for any close to , there exists -linear symmetric and continuous mapping from to such that and
Hence, there exists such that
This and (44) imply that (41) holds. The proof is completed.

#### 5. th Order Necessary Conditions for Strongly -Quasiconvex Functions

We recall that a function from to is quasiconvex if, for every and for every , one has . Inspired by this, we introduce the notion of strong -quasiconvexity for functions from to . A function from to is said to be strongly -quasiconvex if, for every and for every , one has Using the generalized Hessian (see [20]), Luc [1] gave a second-order criterion for quasiconvex functions. We will give a th order necessary codition for a function to be strongly -quasiconvex.

Theorem 9. *Let be a strongly -quasiconvex function from to , an odd number and the closed pointed ordering cone. Then, for any with , there exists such that . *

*Proof. *Suppose that the conclusion is not true. Then, there exist some , with such that . Since is open and is compact, there exists such that
Since is upper continuous, for the previous , there exists such that
for any . Noting that is closed convex, from (48) and (49), we have
From Theorem 2, for any , we can take such that
Noting that is even, we have and , for all .

On the other hand, since is -quasiconvex and , one has or . This is a contradiction.

If and , then . We have the following.

Corollary 10 (see [1]). *Let be a quasiconvex function from to and an odd number. Then, for any with , one has , where .*

#### Acknowledgments

This research was supported by the National Natural Science Foundations, China (Grant no. 11061039, 11061038, and 11261067), an internal Grant of Hong Kong Polytechnic University (G-YF17), and IRTSTYN.