On a Nonsmooth Vector Optimization Problem with Generalized Cone Invexity

Abstract

By using Clarke’s generalized gradients we consider a nonsmooth vector optimization problem with cone constraints and introduce some generalized cone-invex functions called K-α-generalized invex, K-α-nonsmooth invex, and other related functions. Several sufficient optimality conditions and Mond-Weir type weak and converse duality results are obtained for this problem under the assumptions of the generalized cone invexity. The results presented in this paper generalize and extend the previously known results in this area.

1. Introduction

In optimization theory, convexity plays a key role in many aspects of mathematical programming including sufficient optimality conditions and duality theorems; see [1, 2]. Many attempts have been made during the past several decades to relax convexity requirement; see [3–7]. In this endeavor, Hanson [8] introduced invex functions and studied some applications to optimization problem. Subsequently, many authors further weakened invexity hypotheses to establish optimality conditions and duality results for various mathematical programming problems; see, for example, [9–11] and the references cited therein.

Above all, Yen and Sach [12] introduced cone-generalized invex and cone-nonsmooth invex functions. Giorgi and Guerraggio [13] presented the notions of 𝛼-𝐾-invex, 𝛼-𝐾 pseudo-invex, and 𝛼-K quasi-invex functions in the differentiable case and derived optimality and duality results for a vector optimization problem over cones. Khurana[14] extended pseudoinvex functions to differentiable cone-pseudoinvex and strongly cone-pseudoinvex functions. Based on this, Suneja et al. [15] defined cone-nonsmooth quasi-invex, cone-nonsmooth pseudoinvex, and other related functions in terms of Clarke’s [16] generalized directional derivatives and established optimality and duality results for a nonsmooth vector optimization problem.

On the other hand, Noor [17] proposed several classes of 𝛼-invex functions and investigated some properties of the 𝛼-preinvex functions and their differentials. Mishra et al. [18] defined strict pseudo-𝛼-invex and quasi-𝛼-invex functions. Mishra et al. [19] further introduced the concepts of nonsmooth pseudo-𝛼-invex functions and established a relationship between vector variational-like inequality and nonsmooth vector optimization problems by using the nonsmooth 𝛼-invexity.

In the present paper, by using Clarke's generalized gradients of locally Lipschitz functions we are concerned with a nonsmooth vector optimization problem with cone constraints and introduce several generalized invex functions over cones namely 𝐾-𝛼-generalized invex, 𝐾-𝛼-nonsmooth invex, and other related functions, which, respectively, extend some corresponding concepts of [12, 13, 15, 17]. Some sufficient optimality conditions for this problem are obtained by using the above defined concepts. Furthermore, a Mond-Weir type dual is formulated and a few weak and converse duality results are established. We generalize and extend some results presented in the literatures on this topic.

2. Preliminaries and Definitions

Throughout this paper, let 𝜂∶𝑅^𝑛×𝑅^𝑛→𝑅^𝑛 and 𝛼∶𝑅^𝑛×𝑅^𝑛→𝑅₊⧵{0} be two fixed mappings. int𝐾 and 𝐾 denote the interior and closure of 𝐾⊆𝑅^𝑚, respectively. We always assume that 𝐾 is a closed convex cone with int𝐾≠∅.

The positive dual cone 𝐾⁺ of 𝐾 is defined as 𝐾⁺=𝑥^∗∈𝑅^𝑚∶𝑥^∗,𝑥≥0,∀𝑥∈𝐾. (2.1) The strict positive dual cone 𝐾⁺⁺ of 𝐾 is given by 𝐾⁺⁺=𝑥^∗∈𝑅^𝑚∶𝑥^∗,𝑥>0,∀𝑥∈𝐾⧵{0}. (2.2) The following property is from [20], which will be used in the sequel.

Lemma 2.1 (see [20]). Let 𝐾⊆𝑅^𝑚 be a convex cone with int𝐾≠∅. Then,(a)∀𝑢^∗∈𝐾⁺⧵{0},𝑥∈int𝐾⇒⟨𝑢^∗,𝑥⟩>0; (b)∀𝑢^∗∈int𝐾⁺,𝑥∈𝐾⧵{0}⇒⟨𝑢^∗,𝑥⟩>0.

A function 𝜓∶𝑅^𝑛→𝑅 is called locally Lipschitz at 𝑢∈𝑅^𝑛, if there exists 𝑙>0 such that 𝜓(𝑥)−𝜓𝑦≤𝑙𝑥−𝑦, (2.3) for all 𝑥,𝑦 in a neighbourhood of 𝑢.

A function 𝜓 is called locally Lipschitz on 𝑅^𝑛, if it is locally Lipschitz at each point of 𝑅^𝑛.

Definition 2.2 (see [16]). Let 𝜓∶𝑅^𝑛→𝑅 be a locally Lipschitz function, then 𝜓^∘(𝑢;𝑣) denotes Clarke's generalized directional derivative of 𝜓 at 𝑢∈𝑅^𝑛 in the direction 𝑣 and is defined as 𝜓^∘(𝑢;𝑣)=limsup_{𝑦→𝑢𝑡→0}𝜓𝑦+𝑡𝑣−𝜓𝑦𝑡. (2.4) Clarke's generalized gradient of 𝜓 at 𝑢 is denoted by 𝜕𝜓(𝑢) and is defined as 𝜕𝜓(𝑢)=𝜉∈𝑅^𝑛∣𝜓^∘(𝑢;𝑣)≥𝜉,𝑣,∀𝑣∈𝑅^𝑛. (2.5) Let 𝑓∶𝑅^𝑛→𝑅^𝑚 be a vector-valued function given by 𝑓=(𝑓₁,𝑓₂,…,𝑓_𝑚), where 𝑓_𝑖∶𝑅^𝑛→𝑅,𝑖=1,2,…,𝑚. Then 𝑓 is said to be locally Lipschitz on 𝑅^𝑛 if each 𝑓_𝑖 is locally Lipschitz on 𝑅^𝑛. The generalized directional derivative of a locally Lipschitz function 𝑓∶𝑅^𝑛→𝑅^𝑚 at 𝑢∈𝑅^𝑛 in the direction 𝑣 is given by 𝑓^∘(𝑢;𝑣)=𝑓^∘₁(𝑢;𝑣),𝑓^∘₂(𝑢;𝑣),…,𝑓^∘_𝑚(𝑢;𝑣). (2.6) The generalized gradient of 𝑓 at 𝑢 is the set 𝜕𝑓(𝑢)=𝜕𝑓₁(𝑢)×𝜕𝑓₂(𝑢)×⋯×𝜕𝑓_𝑚(𝑢), (2.7) where 𝜕𝑓_𝑖(𝑢)(𝑖=1,2,…,𝑚) is the generalized gradient of 𝑓_𝑖 at 𝑢.
Every 𝐴=(𝑎₁,𝑎₂,…,𝑎_𝑚)∈𝜕𝑓(𝑢) is a continuous linear operator from 𝑅^𝑛 to 𝑅^𝑚 and 𝐴𝑢=𝑎₁,𝑢,𝑎₂,𝑢,…,𝑎_𝑚,𝑢∈𝑅^𝑚,∀𝑢∈𝑅^𝑛. (2.8)

Lemma 2.3 (see [16]). (a) If 𝑓_𝑖∶𝑅^𝑛→𝑅 is locally Lipschitz then, for each 𝑢∈𝑅^𝑛, 𝑓^∘_𝑖(𝑢;𝑣)=max𝜉,𝑣∣𝜉∈𝜕𝑓_𝑖(𝑢),∀𝑣∈𝑅^𝑛,𝑖=1,2,…,𝑚. (2.9)
(b) Let 𝑓_𝑖(𝑖=1,2,…,𝑚) be a finite family of locally Lipschitz functions on 𝑅^𝑛, then ∑^𝑚_𝑖=1𝑓_𝑖 is also locally Lipschitz and 𝜕^𝑚_𝑖=1𝑓_𝑖(𝑢)⊆^𝑚_𝑖=1𝜕𝑓_𝑖(𝑢),∀𝑢∈𝑅^𝑛. (2.10)

Definition 2.4 (see [17]). A function ℎ∶𝑅^𝑛→𝑅 is said to be 𝛼-invex function at 𝑢∈𝑅^𝑛 with respect to 𝛼 and 𝜂, if there exist functions 𝛼 and 𝜂 such that, for every 𝑥∈𝑅^𝑛, we have ℎ(𝑥)−ℎ(𝑢)≥𝛼(𝑥,𝑢)∇ℎ(𝑢),𝜂(𝑥,𝑢). (2.11) In this paper, we consider the following vector optimization problem with cone constraints: (𝑥)(𝑥)𝐾−min𝑓s.t.−𝑔∈𝑄, (VP) where 𝑓∶𝑅^𝑛→𝑅^𝑚,𝑔∶𝑅^𝑛→𝑅^𝑝 are locally Lipschitz functions on 𝑅^𝑛 and 𝐾,𝑄 are closed convex cones with nonempty interiors in 𝑅^𝑚 and 𝑅^𝑝, respectively.
Denote 𝑋={𝑥∈𝑅^𝑛∶−𝑔(𝑥)∈𝑄} the feasible set of problem (VP).
For each 𝜆∈𝐾⁺ and 𝜇∈𝑄⁺, we suppose that 𝜆𝑓=𝜆∘𝑓 and 𝜇𝑔=𝜇∘𝑔 are locally Lipschitz.
Now, we present the concepts of solutions for problem (VP) in the following sense.

Definition 2.5. Let 𝑢∈𝑋, then (a)𝑢 is said to be a minimum of (VP) if for all 𝑥∈𝑋, 𝑓(𝑢)−𝑓(𝑥)∉𝐾⧵{0}; (2.12)(b)𝑢 is said to be a weak minimum of (VP) if for all 𝑥∈𝑋, 𝑓(𝑢)−𝑓(𝑥)∉int𝐾; (2.13)(c)𝑢 is said to be a strong minimum of (VP) if for all 𝑥∈𝑋, 𝑓(𝑥)−𝑓(𝑢)∈𝐾. (2.14)Based on the lines of Yen and Sach [12] and Noor [17], we define the notions as follows.

Definition 2.6. Let 𝑓∶𝑅^𝑛→𝑅^𝑚 be a locally Lipschitz function. 𝑓 is said to be 𝐾-𝛼-generalized invex at 𝑢∈𝑅^𝑛, if there exist functions 𝛼 and 𝜂 such that for every 𝑥∈𝑅^𝑛 and 𝐴∈𝜕𝑓(𝑢), 𝑓(𝑥)−𝑓(𝑢)−𝛼(𝑥,𝑢)𝐴𝜂(𝑥,𝑢)∈𝐾. (2.15)

Definition 2.7. Let 𝑓∶𝑅^𝑛→𝑅^𝑚 be a locally Lipschitz function. 𝑓 is said to be 𝐾-𝛼-nonsmooth invex at 𝑢∈𝑅^𝑛, if there exist functions 𝛼 and 𝜂 such that for every 𝑥∈𝑅^𝑛, 𝑓(𝑥)−𝑓(𝑢)−𝛼(𝑥,𝑢)𝑓^∘𝑢;𝜂∈𝐾, (2.16) where 𝑓^∘𝑢;𝜂=𝑓^∘₁𝑢;𝜂,𝑓^∘₂𝑢;𝜂,…,𝑓^∘_𝑚𝑢;𝜂. (2.17)

Remark 2.8. If 𝑚=1,𝐾=𝑅₊, and 𝑓 is differentiable, then 𝐾-𝛼-generalized invex and 𝐾-𝛼-nonsmooth invex functions become 𝛼-invex function [17]; if 𝛼(𝑥,𝑢)≡1 for all 𝑥,𝑢∈𝑅^𝑛, then 𝐾-𝛼-generalized invex and 𝐾-𝛼-nonsmooth invex functions reduce to 𝐾-generalized invex and 𝐾-nonsmooth invex functions defined by Yen and Sach [12].

Lemma 2.9. If 𝑓 is K-𝛼-generalized invex at 𝑢 with respect to 𝛼 and 𝜂, then 𝑓 is K-𝛼-nonsmooth invex at 𝑢 with respect to the same 𝛼 and 𝜂.

Proof. Since 𝑓 is 𝐾-𝛼-generalized invex at 𝑢, then there exist 𝛼 and 𝜂 such that for every 𝑥∈𝑅^𝑛 and 𝐴∈𝜕𝑓(𝑢)𝑓(𝑥)−𝑓(𝑢)−𝛼(𝑥,𝑢)𝐴𝜂(𝑥,𝑢)∈𝐾. (2.18) By Lemma 2.3, for each 𝑖∈{1,2,…,𝑚}, we choose 𝑎_𝑖∈𝜕𝑓_𝑖(𝑢) such that 𝑎_𝑖,𝜂=max𝑎_𝑖,𝜂∶𝑎_𝑖∈𝜕𝑓_𝑖(𝑢)=𝑓^∘_𝑖𝑢;𝜂. (2.19) Then 𝐴=(𝑎₁,…,𝑎_𝑚)∈𝜕𝑓(𝑢) and 𝑓(𝑥)−𝑓(𝑢)−𝛼(𝑥,𝑢)𝐴𝜂(𝑥,𝑢)∈𝐾, (2.20) equivalently, 𝑓(𝑥)−𝑓(𝑢)−𝛼(𝑥,𝑢)𝑓^∘𝑢;𝜂∈𝐾. (2.21) Hence, 𝑓 is 𝐾-𝛼-nonsmooth invex at 𝑢 with respect to the same 𝛼 and 𝜂.

The following example shows that converse of the above lemma is not true.

Example 2.10. Let 𝐾={(𝑥,𝑦)∣𝑦≤−𝑥} be a cone in 𝑅². Assume that 𝑓=(𝑓₁,𝑓₂), where 𝑓₁(𝑥)=2−𝑥²𝑓,𝑥≥0,2,𝑥<0,²(𝑥)=−𝑥+1,𝑥≥0,1,𝑥<0. (2.22) Let 𝛼∶𝑅×𝑅→𝑅₊⧵{0} and 𝜂∶𝑅×𝑅→𝑅 be defined as 𝛼(𝑥,𝑢)=1/2 and 𝜂(𝑥,𝑢)=2𝑥²−𝑢³, respectively. Then at 𝑢=0, 𝑓(𝑥)(0)−1−𝑓2𝑓^∘0;𝜂(𝑥,0)=−𝑥²,−𝑥+𝑥²(),𝑥≥0,0,0,𝑥<0,∈𝐾. (2.23) Hence, 𝑓 is 𝐾-1/2-nonsmooth invex at 𝑢=0.
It is easy to verify 𝜕𝑓(0)={(0,𝑣)∣−1≤𝑣≤0}.
Taking 𝐴=(0,−1/4)∈𝜕𝑓(0) and 𝑥=−1, we have 𝑓()(0)−1−1−𝑓2()=1𝐴𝜂−1,00,4∉𝐾. (2.24) Therefore, 𝑓 is not 𝐾-1/2-generalized invex at 𝑢=0.
Next, we introduce several related functions of 𝐾-𝛼-nonsmooth invex.

Definition 2.11. 𝑓 is said to be 𝐾-𝛼-nonsmooth quasi-invex at 𝑢∈𝑅^𝑛, if there exist functions 𝛼 and 𝜂 such that for every 𝑥∈𝑅^𝑛, 𝑓(𝑥)−𝑓(𝑢)∉int𝐾⇒−𝛼(𝑥,𝑢)𝑓^∘𝑢;𝜂(𝑥,𝑢)∈𝐾. (2.25)

Definition 2.12. 𝑓 is said to be 𝐾-𝛼-nonsmooth pseudo-invex at 𝑢∈𝑅^𝑛, if there exist functions 𝛼 and 𝜂 such that for every 𝑥∈𝑅^𝑛, 𝑓(𝑢)−𝑓(𝑥)∈int𝐾⇒−𝛼(𝑥,𝑢)𝑓^∘𝑢;𝜂(𝑥,𝑢)∈int𝐾. (2.26)

Definition 2.13. 𝑓 is said to be strict 𝐾-𝛼-nonsmooth pseudo-invex at 𝑢∈𝑅^𝑛, if there exist functions 𝛼 and 𝜂 such that for every 𝑥∈𝑅^𝑛, 𝑓(𝑢)−𝑓(𝑥)∈𝐾⇒−𝛼(𝑥,𝑢)𝑓^∘𝑢;𝜂(𝑥,𝑢)∈int𝐾. (2.27)

Definition 2.14. 𝑓 is said to be strong 𝐾-𝛼-nonsmooth pseudo-invex at 𝑢∈𝑅^𝑛, if there exist functions 𝛼 and 𝜂 such that for every 𝑥∈𝑅^𝑛, 𝑓(𝑥)−𝑓(𝑢)∉𝐾⇒−𝛼(𝑥,𝑢)𝑓^∘𝑢;𝜂(𝑥,𝑢)∈int𝐾. (2.28)

Remark 2.15. If 𝛼(𝑥,𝑢)≡1 for all 𝑥,𝑢∈𝑅^𝑛 and 𝑓 is differentiable, then 𝐾-𝛼-nonsmooth pseudo-invex and strong 𝐾-𝛼-nonsmooth pseudo-invex functions reduce to 𝐾-pseudo-invex and strong 𝐾-pseudo-invex functions, defined by Khurana [14].

Remark 2.16. If 𝑚=1,𝐾=𝑅₊,𝛼(𝑥,𝑢)≡1 for all 𝑥,𝑢∈𝑅^𝑛, and 𝑓 is differentiable, then 𝐾-𝛼-nonsmooth quasi-invex functions reduce to quasi-invex functions and 𝐾-𝛼-nonsmooth pseudo-invex and strong 𝐾-𝛼-nonsmooth pseudo-invex functions reduce to pseudo-invex functions [8].

Remark 2.17. If 𝛼(𝑥,𝑢)≡1 for all 𝑥,𝑢∈𝑅^𝑛, then the above definitions reduce to the corresponding definitions [15]. If 𝑓 is differentiable, then 𝐾-𝛼-generalized invex and 𝐾-𝛼-nonsmooth pseudo-invex functions reduce to 𝛼-𝐾-invex and 𝛼-𝐾 pseudo-invex functions [13], respectively.

3. Optimality Criteria

In this section, we establish a few sufficient optimality conditions for problem (𝑉𝑃) by using the above defined functions.

Theorem 3.1. Let 𝑓 be 𝐾-𝛼-generalized invex and 𝑔 be 𝑄-𝛼-generalized invex at 𝑢∈𝑋 with respect to the same 𝛼 and 𝜂. We assume that there exist 𝜆∈𝐾⁺,𝜆≠0,𝜇∈𝑄⁺ such that 0∈𝜕𝜆𝑓(𝑢)+𝜕𝜇𝑔(𝑢),𝜇𝑔(𝑢)=0. (3.1)(3.2) Then 𝑢 is a weak minimum of (VP).

Proof. By contradiction, we assume that 𝑢 is not a weak minimum of (VP). Then there exists a feasible solution 𝑥 of (VP) such that 𝑓(𝑢)−𝑓(𝑥)∈int𝐾. (3.3) From (3.1), it follows that there exist 𝑠∈𝜕(𝜆𝑓)(𝑢) and 𝑡∈𝜕(𝜇𝑔)(𝑢) such that 𝑠+𝑡=0. (3.4) Since 𝑓 is 𝐾-𝛼-generalized invex and 𝑔 is 𝑄-𝛼-generalized invex at 𝑢, we get 𝑓(𝑥)−𝑓(𝑢)−𝛼(𝑥,𝑢)𝐴𝜂(𝑥,𝑢)∈𝐾,∀𝐴∈𝜕𝑓(𝑢),𝑔(𝑥)−𝑔(𝑢)−𝛼(𝑥,𝑢)𝐵𝜂(𝑥,𝑢)∈𝑄,∀𝐵∈𝜕𝑔(𝑢). (3.5)(3.6) Summing (3.3) and (3.5), we have −𝛼(𝑥,𝑢)𝐴𝜂(𝑥,𝑢)∈int𝐾,∀𝐴∈𝜕𝑓(𝑢). (3.7) As 𝜆∈𝐾⁺,𝜆≠0,  from Lemma 2.1, we obtain 𝛼(𝑥,𝑢)𝜆𝐴𝜂(𝑥,𝑢)<0,∀𝐴∈𝜕𝑓(𝑢), (3.8) which yields 𝛼(𝑥,𝑢)𝑠𝜂(𝑥,𝑢)<0,as𝜆≠0,𝑠∈𝜕𝜆𝑓(𝑢)=𝜆𝜕𝑓(𝑢). (3.9) Considering positivity of 𝛼(𝑥,𝑢) and (3.4), one has 𝑡𝜂(𝑥,𝑢)>0. (3.10) From 𝑡∈𝜕(𝜇𝑔)(𝑢)=𝜇𝜕𝑔(𝑢), we deduce 𝑡=𝜇𝐵^∗,forsome𝐵^∗∈𝜕𝑔(𝑢). (3.11) Hence, 𝜇𝐵^∗𝜂(𝑥,𝑢)>0,where𝐵^∗∈𝜕𝑔(𝑢). (3.12) By 𝜇∈𝑄⁺, relation (3.6) gives 𝜇𝑔(𝑥)−𝜇𝑔(𝑢)−𝜇𝛼(𝑥,𝑢)𝐵𝜂(𝑥,𝑢)≥0,∀𝐵∈𝜕𝑔(𝑢). (3.13) By virtue of (3.2) and 𝑥∈𝑋, the above inequality implies −𝜇𝛼(𝑥,𝑢)𝐵𝜂(𝑥,𝑢)≥0,∀𝐵∈𝜕𝑔(𝑢), (3.14) that is, 𝜇𝐵𝜂(𝑥,𝑢)≤0,∀𝐵∈𝜕𝑔(𝑢), (3.15) which is a contradiction to (3.12).
Therefore, 𝑢 is a weak minimum of (VP).

Theorem 3.2. Let 𝑓 be K-𝛼-generalized invex and 𝑔 be 𝑄-𝛼-generalized invex at 𝑢∈𝑋 with respect to the same 𝛼 and 𝜂. We assume that there exist 𝜆∈𝐾⁺⁺,𝜇∈𝑄⁺ such that (3.1) and (3.2) hold. Then 𝑢 is a minimum of (VP).

Proof. Assume contrary to the result that 𝑢 is not a minimum of (VP). Then there exists 𝑥∈𝑋 such that 𝑓(𝑢)−𝑓(𝑥)∈𝐾⧵{0}. (3.16) From (3.1), it follows that there exist 𝑠∈𝜕(𝜆𝑓)(𝑢) and 𝑡∈𝜕(𝜇𝑔)(𝑢) such that 𝑠+𝑡=0. (3.17) Since 𝑓 is 𝐾-𝛼-generalized invex at 𝑢∈𝑋, we get 𝑓(𝑥)−𝑓(𝑢)−𝛼(𝑥,𝑢)𝐴𝜂(𝑥,𝑢)∈𝐾,∀𝐴∈𝜕𝑓(𝑢). (3.18) Utilizing (3.16), we deduce −𝛼(𝑥,𝑢)𝐴𝜂(𝑥,𝑢)∈𝐾⧵{0},∀𝐴∈𝜕𝑓(𝑢). (3.19) According to 𝜆∈𝐾⁺⁺, we obtain 𝛼(𝑥,𝑢)𝜆𝐴𝜂(𝑥,𝑢)<0,∀𝐴∈𝜕𝑓(𝑢). (3.20) Next proceeding on the same lines as in the proof of Theorem 3.1, we obtain a contradiction.
Thus, 𝑢 is a minimum of (VP).

Theorem 3.3. Let 𝑓 be 𝐾-𝛼-nonsmooth pseudo-invex and 𝑔 be 𝑄-𝛼-nonsmooth quasi-invex at 𝑢∈𝑋 with respect to the same 𝛼 and 𝜂. We assume that there exist 𝜆∈𝐾⁺,𝜆≠0,𝜇∈𝑄⁺ such that (3.1) and (3.2) hold. Then 𝑢 is a weak minimum of (VP).

Proof. It follows from (3.1) that there exist 𝑠∈𝜕(𝜆𝑓)(𝑢) and 𝑡∈𝜕(𝜇𝑔)(𝑢) such that 𝑠+𝑡=0. (3.21) Suppose that 𝑢 is not a weak minimum of (VP). Then there exists 𝑥∈𝑋 such that 𝑓(𝑢)−𝑓(𝑥)∈int𝐾. (3.22) Since 𝑓 is 𝐾-𝛼-nonsmooth pseudo-invex at 𝑢∈𝑋, we deduce −𝛼(𝑥,𝑢)𝑓^∘𝑢;𝜂(𝑥,𝑢)∈int𝐾. (3.23) By 𝜆∈𝐾⁺,𝜆≠0 and Lemma 2.1, we obtain 𝛼(𝑥,𝑢)𝜆𝑓^∘𝑢;𝜂(𝑥,𝑢)<0. (3.24) From 𝛼(𝑥,𝑢)>0 and 𝑓^∘_𝑖(𝑢;𝜂(𝑥,𝑢))=max{⟨𝑣_𝑖,𝜂⟩∶𝑣_𝑖∈𝜕𝑓_𝑖(𝑢)},𝑖=1,2,…,𝑚, we have 𝜆𝐴𝜂(𝑥,𝑢)<0,∀𝐴∈𝜕𝑓(𝑢), (3.25) which implies, 𝑠𝜂(𝑥,𝑢)<0,where𝑠∈𝜕𝜆𝑓(𝑢)=𝜆𝜕𝑓(𝑢). (3.26) By 𝑥∈𝑋 and 𝜇∈𝑄⁺,  −𝑔(𝑥)∈𝑄 gives 𝜇𝑔(𝑥)≤0. (3.27) Taking (3.2) into account, one has 𝜇𝑔(𝑥)−𝑔(𝑢)≤0. (3.28) Next we prove 𝜇𝑔^∘𝑢;𝜂(𝑥,𝑢)≤0. (3.29) If 𝜇=0, inequality (3.29) holds obviously.
If 𝜇≠0, from (3.28) and Lemma 2.1, we deduce 𝑔(𝑥)−𝑔(𝑢)∉int𝑄. (3.30) Since 𝑔 is Q-𝛼-nonsmooth quasi-invex at 𝑢∈𝑋, we have −𝛼(𝑥,𝑢)𝑔^∘𝑢;𝜂(𝑥,𝑢)∈𝑄. (3.31) From 𝛼(𝑥,𝑢)>0 and 𝜇∈𝑄⁺, it follows that (3.29) also holds.
Similarly, by Lemma 2.3, inequality (3.29) gives 𝜇𝐵𝜂(𝑥,𝑢)≤0,∀𝐵∈𝜕𝑔(𝑢), (3.32) which yields, 𝑡𝜂(𝑥,𝑢)≤0,where𝑡∈𝜕𝜇𝑔(𝑢)=𝜇𝜕𝑔(𝑢). (3.33) Hence, 𝑠𝜂(𝑥,𝑢)≥0, (3.34) which is in contradiction with (3.26).
Therefore, 𝑢 is a weak minimum of (VP).

The following example illustrates the above theorem.

Example 3.4. Consider the vector optimization problem (VP) where 𝐾={(𝑥,𝑦)𝑦≥−𝑥,𝑦≥𝑥},𝑄={(𝑥,𝑦)−𝑥≤𝑦≤𝑥,𝑥≥0}, and 𝑓_𝑖,𝑔_𝑖∶𝑅→𝑅,𝑖=1,2 are defined as 𝑓₁(𝑥)=−𝑥²𝑓−𝑥,𝑥>0,0,𝑥≤0,²(𝑥)=𝑔−2𝑥+1,𝑥>0,1,𝑥≤0,₁(𝑥)=𝑥³𝑥,𝑥>0,3𝑔,𝑥≤0,²(𝑥)=𝑥²𝑥,𝑥>0,2,𝑥≤0. (3.35) Let 𝛼∶𝑅×𝑅→𝑅₊⧵{0} and 𝜂∶𝑅×𝑅→𝑅 be defined as 𝛼(𝑥,𝑢)=2 and 𝜂(𝑥,𝑢)=(𝑥+2𝑢)³, respectively. It is easily testified that 𝑓 and 𝑔 are 𝐾-𝛼-nonsmooth pseudo-invex and 𝐾-𝛼-nonsmooth quasi-invex at 𝑢=0, respectively. The feasible set of (VP) is given by 𝑋=(−∞,0].
It is also easy to verify 𝜕𝑓(0)=[−1,0]×[−2,0],𝜕𝑔(0)=[0,1/3]×[0,1/2].
Taking 𝜆=(1,3)∈𝐾⁺ and 𝜇=(3,1)∈𝑄⁺, we have 30∈−7,2=𝜕𝜆𝑓(0)+𝜕𝜇𝑔(0),𝜇𝑔(0)=0, (3.36) which imply that (3.1) and (3.2) hold.
Therefore, by Theorem 3.3, 𝑢=0 is a weak minimum of (VP).

Theorem 3.5. Let 𝑓 be strong K-𝛼-nonsmooth pseudo-invex and 𝑔 be Q-𝛼-nonsmooth quasi-invex at 𝑢∈𝑋 with respect to the same 𝛼 and 𝜂. We assume that there exist 𝜆∈𝐾⁺,𝜆≠0,𝜇∈𝑄⁺ such that (3.1) and (3.2) hold. Then 𝑢 is a strong minimum of (VP).

Proof. From (3.1), it follows that there exist 𝑠∈𝜕(𝜆𝑓)(𝑢) and 𝑡∈𝜕(𝜇𝑔)(𝑢) such that 𝑠+𝑡=0. (3.37) Assume that 𝑢 is not a strong minimum of (VP). Then there exists 𝑥∈𝑋 such that 𝑓(𝑥)−𝑓(𝑢)∉𝐾. (3.38) Since 𝑓 is strong 𝐾-𝛼-nonsmooth pseudo-invex at 𝑢, we deduce −𝛼(𝑥,𝑢)𝑓^∘𝑢;𝜂(𝑥,𝑢)∈int𝐾. (3.39) Next proceeding on the same lines as in the proof of Theorem 3.3, we get a contradiction.
Hence 𝑢 is a strong minimum of (VP).

Theorem 3.6. Let 𝑓 be strict K-𝛼-nonsmooth pseudo-invex and 𝑔 be Q-𝛼-nonsmooth quasi-invex at 𝑢∈𝑋 with respect to the same 𝛼 and 𝜂. We assume that there exist 𝜆∈𝐾⁺,𝜆≠0,𝜇∈𝑄⁺ such that (3.1) and (3.2) hold. Then 𝑢 is a minimum of (VP).

Proof. From (3.1), it follows that there exist 𝑠∈𝜕(𝜆𝑓)(𝑢) and 𝑡∈𝜕(𝜇𝑔)(𝑢) such that 𝑠+𝑡=0. (3.40) By contradiction, assume that 𝑢 is not a minimum of (VP). Then there exists 𝑥∈𝑋 such that 𝑓(𝑢)−𝑓(𝑥)∈𝐾⧵{0}⊂𝐾. (3.41) Since 𝑓 is strict 𝐾-𝛼-nonsmooth pseudo-invex at 𝑢, we have −𝛼(𝑥,𝑢)𝑓^∘𝑢;𝜂(𝑥,𝑢)∈int𝐾. (3.42) Next as in Theorem 3.3 we arrive at a contradiction.
Therefore, 𝑢 is a minimum of (VP).

4. Duality

In relation to (VP), we consider the following Mond-Weir type dual problem: 𝑦𝑦𝑦,𝑦𝐾−max𝑓s.t.0∈𝜕𝜆𝑓+𝜕𝜇𝑔𝜇𝑔≥0,𝑦∈𝑅^𝑛,𝜆∈𝐾⁺,𝜆≠0,𝜇∈𝑄⁺. (VD) Denote the feasible set of problem (VD) by 𝐺, namely, 𝐺={(𝑦,𝜆,𝜇)∶0∈𝜕(𝜆𝑓)(𝑦)+𝜕(𝜇𝑔)(𝑦),𝜇𝑔(𝑦)≥0,𝑦∈𝑅^𝑛,𝜆∈𝐾⁺,𝜆≠0,𝜇∈𝑄⁺}.

Now, we establish weak and converse duality results.

Theorem 4.1 (Weak duality). Let 𝑥∈𝑋 and (𝑦,𝜆,𝜇)∈𝐺. If 𝑓 is K-𝛼-nonsmooth pseudo-invex and 𝑔 is Q-𝛼-nonsmooth quasi-invex at 𝑦 with respect to the same 𝛼 and 𝜂, then 𝑓𝑦−𝑓(𝑥)∉int𝐾. (4.1)

Proof. Since (𝑦,𝜆,𝜇)∈𝐺, from (VD), it follows that there exist 𝑠∈𝜕(𝜆𝑓)(𝑦) and 𝑡∈𝜕(𝜇𝑔)(𝑦) such that 𝑠+𝑡=0. (4.2) By contradiction, we assume that 𝑓(𝑦)−𝑓(𝑥)∈int𝐾.
Since 𝑓 is 𝐾-𝛼-nonsmooth pseudo-invex at 𝑦, we have −𝛼𝑥,𝑦𝑓^∘𝑦;𝜂𝑥,𝑦∈int𝐾. (4.3) By 𝜆∈𝐾⁺,𝜆≠0 and Lemma 2.1, we get 𝛼𝑥,𝑦𝜆𝑓^∘𝑦;𝜂𝑥,𝑦<0. (4.4) From 𝛼(𝑥,𝑦)>0 and Lemma 2.3, we deduce 𝜆𝐴𝜂𝑥,𝑦<0,∀𝐴∈𝜕𝑓𝑦, (4.5) which yields 𝑠𝜂𝑥,𝑦<0,where𝑠∈𝜕𝜆𝑓𝑦=𝜆𝜕𝑓𝑦. (4.6) Using (4.2), we obtain 𝑡𝜂𝑥,𝑦>0. (4.7) From 𝑡∈𝜕(𝜇𝑔)(𝑦)=𝜇𝜕𝑔(𝑦), it follows that there exists 𝐵^∗∈𝜕𝑔(𝑦) such that 𝑡=𝜇𝐵^∗. (4.8) Hence, 𝜇𝐵^∗𝜂𝑥,𝑦>0,where𝐵^∗∈𝜕𝑔𝑦. (4.9) From 𝑥∈𝑋 and (𝑦,𝜆,𝜇)∈𝐺, we find that 𝜇𝑔(𝑥)≤0≤𝜇𝑔𝑦. (4.10) Now we claim that 𝜇𝑔^∘𝑦;𝜂𝑥,𝑦≤0. (4.11)
If 𝜇=0, then (4.11) holds trivially.
If 𝜇≠0, from (4.10) and Lemma 2.1 we have 𝑔(𝑥)−𝑔𝑦∉int𝑄. (4.12) As 𝑔 is 𝑄-𝛼-nonsmooth quasi-invex at 𝑦, we obtain −𝛼𝑥,𝑦𝑔^∘𝑦;𝜂𝑥,𝑦∈𝑄, (4.13) which means that (4.11) also holds and is equivalent to 𝜇𝐵𝜂𝑥,𝑦≤0,∀𝐵∈𝜕𝑔𝑦, (4.14) which is a contradiction to (4.9). Thus 𝑓𝑦−𝑓(𝑥)∉int𝐾. (4.15)

Theorem 4.2 (Weak duality). Let 𝑥∈𝑋 and (𝑦,𝜆,𝜇)∈𝐺. If 𝑓 is K-𝛼-generalized invex and 𝑔 is Q-𝛼-generalized invex at 𝑦 with respect to the same 𝛼 and 𝜂, then 𝑓𝑦−𝑓(𝑥)∉int𝐾. (4.16)

Proof. Since (𝑦,𝜆,𝜇)∈𝐺, from (VD), it follows that there exist 𝑠∈𝜕(𝜆𝑓)(𝑦) and 𝑡∈𝜕(𝜇𝑔)(𝑦) such that 𝑠+𝑡=0. (4.17) We assume contrary to the result that 𝑓𝑦−𝑓(𝑥)∈int𝐾. (4.18) Since 𝑓 is 𝐾-𝛼-generalized invex and 𝑔 is Q-𝛼-generalized invex at 𝑦, we get 𝑓(𝑥)−𝑓𝑦−𝛼𝑥,𝑦𝐴𝜂𝑥,𝑦∈𝐾,∀𝐴∈𝜕𝑓𝑦,𝑔(𝑥)−𝑔𝑦−𝛼𝑥,𝑦𝐵𝜂𝑥,𝑦∈𝑄,∀𝐵∈𝜕𝑔𝑦. (4.19)(4.20) Summing (4.18) and (4.19), we have −𝛼𝑥,𝑦𝐴𝜂𝑥,𝑦∈int𝐾,∀𝐴∈𝜕𝑓𝑦. (4.21) By 𝜆∈𝐾⁺,𝜆≠0 and Lemma 2.1, we obtain 𝛼𝑥,𝑦𝜆𝐴𝜂𝑥,𝑦<0, (4.22) which yields 𝛼𝑥,𝑦𝑠𝜂𝑥,𝑦<0,where𝑠∈𝜕𝜆𝑓𝑦=𝜆𝜕𝑓𝑦,𝜆≠0. (4.23) Applying positivity of 𝛼(𝑥,𝑦) and (4.17), we get 𝑡𝜂𝑥,𝑦>0. (4.24) By the fact that 𝑡∈𝜕(𝜇𝑔)(𝑦)=𝜇𝜕𝑔(𝑦), one has 𝑡=𝜇𝐵^∗,where𝐵^∗∈𝜕𝑔𝑦. (4.25) Hence, 𝜇𝐵^∗𝜂𝑥,𝑦>0,where𝐵^∗∈𝜕𝑔𝑦. (4.26) From 𝜇∈𝑄⁺ and (4.20), we obtain 𝜇𝑔(𝑥)−𝜇𝑔𝑦−𝜇𝛼𝑥,𝑦𝐵𝜂𝑥,𝑦≥0,∀𝐵∈𝜕𝑔𝑦. (4.27) As 𝑥∈𝑋 and (𝑦,𝜆,𝜇)∈𝐺, we get 𝜇𝑔(𝑥)≤0≤𝜇𝑔𝑦. (4.28) Using the above relation, (4.27) yields −𝜇𝛼𝑥,𝑦𝐵𝜂𝑥,𝑦≥0,∀𝐵∈𝜕𝑔𝑦, (4.29) that is, 𝜇𝐵𝜂𝑥,𝑦≤0,∀𝐵∈𝜕𝑔𝑦, (4.30) which contradicts (4.26). Therefore, 𝑓𝑦−𝑓(𝑥)∉int𝐾. (4.31)

Theorem 4.3 (Converse duality). Let 𝑦∈𝑋 and (𝑦,𝜆,𝜇)∈𝐺. Assume that 𝑓 is K-𝛼-nonsmooth pseud-invex and 𝑔 is Q-𝛼-nonsmooth quasi-invex at 𝑦 with respect to the same 𝛼 and 𝜂. Then 𝑦 is a weak minimum of (VP).

Proof. Since (𝑦,𝜆,𝜇)∈𝐺, from (VD), it follows that there exist 𝑠∈𝜕(𝜆𝑓)(𝑦) and 𝑡∈𝜕(𝜇𝑔)(𝑦) such that 𝑠+𝑡=0. (4.32) Assume contrary to the result that 𝑦 is not a weak minimum of (VP). Then there exists 𝑥∈𝑋 such that 𝑓𝑦−𝑓(𝑥)∈int𝐾. (4.33) Since 𝑓 is 𝐾-𝛼-nonsmooth pseudo-invex at 𝑦, we have −𝛼𝑥,𝑦𝑓^∘𝑦;𝜂𝑥,𝑦∈int𝐾. (4.34) By 𝜆∈𝐾⁺,𝜆≠0 and Lemma 2.1, we get 𝛼𝑥,𝑦𝜆𝑓^∘𝑦;𝜂𝑥,𝑦<0, (4.35) which is equivalent to 𝜆𝐴𝜂𝑥,𝑦<0,∀𝐴∈𝜕𝑓𝑦, (4.36) which yields 𝑠𝜂𝑥,𝑦<0,where𝑠∈𝜕𝜆𝑓𝑦=𝜆𝜕𝑓𝑦,𝜆≠0. (4.37) Using (4.32), we obtain 𝑡𝜂𝑥,𝑦>0. (4.38) As 𝑡∈𝜕(𝜇𝑔)(𝑦)=𝜇𝜕𝑔(𝑦), thus 𝑡=𝜇𝐵^∗, where 𝐵^∗∈𝜕𝑔(𝑦).
Hence, 𝜇𝐵^∗𝜂𝑥,𝑦>0,where𝐵^∗∈𝜕𝑔𝑦. (4.39) By 𝑥∈𝑋 and (𝑦,𝜆,𝜇)∈𝐺, we have 𝜇𝑔(𝑥)≤0≤𝜇𝑔𝑦. (4.40) By the similar argument to that of Theorem 4.1, we can prove that 𝜇𝑔^∘𝑦;𝜂𝑥,𝑦≤0, (4.41) which is equivalent to 𝜇𝐵𝜂𝑥,𝑦≤0,∀𝐵∈𝜕𝑔𝑦, (4.42) which is in contradiction with (4.39).
Therefore, 𝑦 is a weak minimum of (VP).

Theorem 4.4 (Converse duality). Let 𝑦∈𝑋 and (𝑦,𝜆,𝜇)∈𝐺. Assume that 𝑓 is K-𝛼-generalized invex and 𝑔 is Q-𝛼-generalized invex at 𝑦 with respect to the same 𝛼 and 𝜂. Then 𝑦 is a weak minimum of (VP).

Proof. The proof of the above theorem is very similar to the proof of Theorem 3.1, except that for this case we use the feasibility of (𝑦,𝜆,𝜇) for (VD) instead of the relations (3.1) and (3.2).

Acknowledgments

This work was supported by the National Natural Science Foundation of China (no. 60974082) and supported by Foundation Project of China Chongqing Education Commission.