#### Abstract

We introduce a class of functions called geodesic -preinvex and geodesic -invex functions on Riemannian manifolds and generalize the notions to the so-called geodesic quasi/pseudo -preinvex and geodesic quasi/pseudo -invex functions. We discuss the links among these functions under appropriate conditions and obtain results concerning extremum points of a nonsmooth geodesic -preinvex function by using the proximal subdifferential. Moreover, we study a differentiable multiobjective optimization problem involving new classes of generalized geodesic -invex functions and derive Kuhn-Tucker-type sufficient conditions for a feasible point to be an efficient or properly efficient solution. Finally, a Mond-Weir type duality is formulated and some duality results are given for the pair of primal and dual programming.

#### 1. Introduction

Convex functions play an important role in optimization theory and there are several classes of functions given in the literature with the goal to weaken the limitations of convexity in mathematical programming. Generalized convex functions, labelled as -vex functions, were introduced by Bector and Singh [1]. In 1981, Hanson [2] introduced the concept of invexity and proved that the Kuhn-Tucker conditions are sufficient for optimality of a nonlinear programming problem under invexity conditions. Preinvex functions were defined by Ben-Israel and Mond [3], and, in [4], Weir and Mond showed how and where preinvex functions could replace convex functions in multiple objective optimization problem. These functions were further generalized to pseudo/quasi -vex, -invex, and pseudo/quasi -invex functions by Bector et al. [5] and to -preinvex by Suneja et al. [6]. In [5], Bector et al. obtained sufficient optimality criteria and duality results for a nonlinear programming problem involving -vex and -invex functions. There are also many papers in the literature concerning the generalization of convexity in connection with sufficiency and duality in optimization problems (see, e.g., [7–12] and the references therein).

A manifold is not a linear space and extensions of concepts and techniques from linear spaces to Riemannian manifolds are natural. In the literature many authors studied generalized convex functions and many results in convex analysis and optimization theory were extended to Riemannian manifolds (see [13–28] and the references therein). Rapcsák [27] and Udriste [28] considered a generalization of convexity called geodesic convexity. In this setting the linear space is replaced by a Riemannian manifold and the line segment by a geodesic. Pini [22] introduced the notion of invex function on Riemannian manifolds, while Mititelu [24] investigated its generalization. The concepts of geodesic invex sets, geodesic invex, and preinvex functions on Riemannian manifolds were defined by Barani and Pouryayevali [17]. They established the relation between geodesic invexity and preinvexity of functions, and they also obtained results concerning extremum points of a nonsmooth geodesic preinvex function by using the proximal subdifferential. Subsequently, Agarwal et al. [20] proposed and discussed geodesic -preinvexity on Riemannian manifolds, which generalized the corresponding results studied by Barani and Pouryayevali [17]. A new concept of geodesic roughly -invexity and its generalization on Hadamard manifolds were introduced by Zhou and Huang [26]. They studied the properties of these functions and they established sufficient optimality conditions and duality in nonlinear programming problems.

In this paper, we introduce a class of geodesic -preinvex and -invex functions on Riemannian manifolds and extend them to geodesic quasi/pseudo -preinvex and geodesic quasi/pseudo -invex functions. We discuss the links among these functions under suitable assumptions. By applying the proximal subdifferential, we relax the smoothness condition and study the question of global minima for geodesic -preinvex functions on Riemannian manifolds. As applications, we investigate a multiobjective programming problem involving generalized geodesic -invex functions and derive the Kuhn-Tucker-type sufficient optimality conditions for a feasible point to be an efficient or properly efficient solution. Finally, a Mond-Weir type duality is formulated and some duality results are obtained for the pair of primal and dual programming. The results presented in this paper extend some known results due to Barani and Pouryayevali [17, 23].

#### 2. Preliminaries

In this section, we recall some definitions and known results about Riemannian manifolds which will be used throughout the paper. These can be found in many introductory books on Riemannian geometry, such as in [29–32].

Let be a smooth manifold modelled on a Hilbert space , either finite dimensional or infinite dimensional, endowed with a Riemannian metric on the tangent space . The corresponding norm is denoted by . The tangent bundle of is denoted by , which is naturally a manifold. Given a piecewise path joining to , that is, and , we can define the length of by For any two points , we define Then is a metric on which defines the same topology as the one naturally has as a manifold. For this metric we define the open ball centered at the point with radius ; that is, Let us recall that in every Riemannian manifold there exists exactly one covariant derivation called the Levi-Civita connection denoted by for any vector fields , on . We also recall that a geodesic is a smooth path whose tangent is parallel along the path ; that is, satisfies the equation . Any path joins and in such that is a geodesic and is called a minimal geodesic. The existence theorem for ordinary differential equation implies that for every , there exists an open interval containing and exactly one geodesic with . This implies that there is an open neighborhood of the submanifold of such that for every , the geodesic is defined for (see, e.g., [25]). The exponential map is then defined as , where is the geodesic defined by its position and velocity at .

If is a geodesic, then for each , the Levi-Civita connection induces an isometry , the so-called parallel translation from to along , which is defined by where is the unique vector field satisfying for all and .

Let be a differentiable function. We will denote by the differential at .

We also recall that a simply connected complete Riemannian manifold of nonpositive curvature is called a Cartan-Hadamard manifold.

Barani and Pouryayevali [17] first defined geodesic invex sets and introduced geodesic preinvex functions on Riemannian manifolds.

*Definition 1. *Let be a Riemannian manifold and be a function such that for every . A nonempty subset of is said to be geodesic invex with respect to , if for every , there exists exactly one geodesic such that

*Definition 2. *Let be a Riemannian manifold and a geodesic invex set with respect to . We say that a function is geodesic preinvex if

In 1993, Suneja et al. [6] introduced the generalization of preinvex functions on , and we now improve and extend the definition of -preinvex functions to Riemannian manifolds.

*Definition 3. *Let be a Riemannian manifold and an open invex set with respect to . A function is said to be(i)geodesic -preinvex (GBPIX) at with respect to and if
where with for all and , and is the unique geodesic defined in Definition 1;(ii)GBPIX on with respect to and if it is GBPIX at each with respect to the same and ;(iii)strictly GBPIX (SGBPIX) on with respect to and if inequality (8) is strict for all with ;(iv)geodesic quasi -preinvex (GQBPIX) with respect to and at if
(v)GQBPIX on with respect to and if it is GQBPIX at each with respect to the same and ;(vi)geodesic pseudo -preinvex (GPBPIX) with respect to and if, there exists a strictly positive function such that
(vii)GPBPIX on with respect to and , if it is GPBPIX at each with respect to the same and .

*Remark 4. *By Definition 1, it is clear that SGBPIX GBPIX, GBPIX GQBPIX, and GPBPIX GQBPIX.

*Remark 5. *If , then GBPIX GPBPIX.

*Remark 6. *The above definition of geodesic -preinvexity on Riemannian manifolds is also a generalization of geodesic preinvexity discussed in [17]. It is easy to see that every geodesic preinvex function with respect to is a geodesic -preinvex function with respect to and , where , but the converse is not true, as illustrated in the following example.

*Example 7. *Let . For any with and , let be defined by , let be defined by
and let be defined by
Define a geodesic on as follows:
Then, we can easily verify that is geodesic -preinvex with respect to and , but is not geodesic preinvex with respect to .

In [22], Pini introduced the concept of geodesic invexity on Riemannian manifolds. Motivated by the definitions of (pseudo/quasi) -invex functions on given in [5], we present the following definition.

*Definition 8. *Let be a Riemannian manifold and a nonnegative real function. A differentiable function is said to be(i)geodesic -invex (GBIX) at with respect to and if
(ii)GBIX on with respect to and if it is GBIX at each with respect to the same and ;(iii)strictly geodesic -invex on with respect to and if inequality (14) is strict for all with ;(iv)geodesic quasi -invex (GQBIX) at with respect to and if
(v)GQBIX on with respect to and if it is GQBIX at each with respect to the same and ;(vi)geodesic pseudo -invex (GPBIX) at with respect to and if
or equivalently,
(vii)strictly geodesic pseudo -invex (SGPBIX) at with respect to and if
or equivalently,
(viii)GPBIX/SGBPIX on with respect to and if it is GPBIX/SGBPIX at each with respect to the same and .

*Remark 9. *An invex function with respect to discussed on Riemannian manifolds in [17] is also a GBIX function with respect to and with , but the converse is not true.

*Example 10. *Let be a Riemannian manifold and a differentiable function such that for every , . Let be a bifunction and let be defined by
Then, for every , one has
Therefore, is geodesic -invex with respect to and , but is not geodesic invex with respect to the same whenever .

*Remark 11. *Every geodesic -invex function with respect to and , where , , is geodesic invex with respect to some , where

*Remark 12. *Every geodesic pseudoinvex function with respect to in [18] is geodesic pseudo -invex with respect to the same . However, the converse is not necessarily true when , for some .

*Remark 13. *From Definition 8, we have

Finally we present the following definitions which will be useful in the sequel.

*Definition 14 (see [17]). *Let be a Riemannian manifold. We say that the function satisfies the condition (*C*), if for each , and for the geodesic satisfying , we have(i);
(ii).

*Definition 15 (see [33]). *Let be a Riemannian manifold and a lower semicontinuous function. A point is a proximal subgradient of at if there exist positive numbers and such that
where .

The set of all proximal subgradients of at is denoted by and is called the proximal subdifferential of at .

#### 3. GBPIX (GBIX) Functions and Their Generalization

Theorem 16. *Let be a Riemannian manifold, let be an open invex subset of with respect to , and let be a nonnegative real function. Assume that is a differentiable GBPIX function with respect to and . Then is a GBIX function with respect to and , where .*

*Proof. *Let . Since is a geodesic invex set with respect to , there exists exactly one geodesic such that
Noting that is GBPIX with respect to and , we have
which implies
Dividing by and taking the limit as , we obtain
Therefore, . This completes the proof.

Theorem 17. *Let be a Riemannian manifold and an open geodesic invex set with respect to . Suppose that is differentiable GBIX with respect to and , where . If satisfies condition (C) and , for all , then is GBPIX with respect to and , where
**
for all and , for some .*

*Proof. *Since is geodesic invex with respect to , there exists exactly one geodesic such that
Fix and set . Then we have

Now multiplying (31) and (32) by and , respectively, and adding, then we have
It follows from condition (*C*) that
Therefore,
where
This completes the proof.

Theorem 18. *Let be a Riemannian manifold, let be an open geodesic invex set with respect to , and let be GBPIX with respect to and .*(i)*Every lower section of defined by
is a geodesic invex set with respect to .*(ii)*The set of solutions for problem
is a geodesic invex set with respect to .*(iii)*If is a local optimal solution to the problem (P) and , then is a global minimum for (P). Moreover, if is strictly GBPIX, then the global optimal solution of problem (P) is unique.*

*Proof. *(i) Let . Since is a geodesic invex set with respect to , there exists exactly one geodesic such that
The GBPIX of gives
which implies that for all .

(ii) If has no optimal solution in , then , which is obviously a geodesic invex set. If and is an arbitrary optimal point for (), then , which is also a geodesic invex set with respect to by .

(iii) Suppose that is a local minimum. Then there is a neighborhood such that
If is not a global minimum of , then there exists a point such that
Since is a geodesic invex set with respect to , there exists exactly one geodesic such that
By the continuity of the distance function and the geodesic , there exists a number such that for all . Hence, . It follows from the GBPIX of that
Hence, for each , , which is a contradiction to (41).

If is another global optimal solution for (*P*) and , then . It follows from the strict GBPIX of that
which contradicts the optimality of for (*P*). Therefore, the solution of (*P*) is unique. This completes the proof.

Similar reasoning to that in the proof of Theorem 5.2 in [17] yields the following result.

Theorem 19. *Let be a Cartan-Hadamard manifold and be an open geodesic invex set with respect to with for . Suppose that is a lower semicontinuous GBPIX function with respect to and . Let and . Then there exists a number such that
**
where .*

From Theorem 19, we can obtain the following corollary.

Corollary 20. *Let be a Cartan-Hadamard manifold and be an open geodesic invex set with respect to with for . Suppose that is a lower semicontinuous geodesic -preinvex function with respect to and and . Let and . Then is a global minimum of .*

*Remark 21. *It should be noted that if is a subset of a Riemannian manifold and is a lower semicontinuous function which has a local minimum at , then (see [33]).

*Remark 22. *Theorems 16, 17, 18, and 19 extend not only the corresponding results from to Riemannian manifolds, but also Theorems 4.1, 4.2, 5.1, and 5.2 of [17], respectively.

The following theorems reveal the relations among geodesic quasi -preinvexity, geodesic quasi -invexity, geodesic pseudo -preinvexity, and geodesic pseudo -invexity for a differentiable function .

Theorem 23. *Let be a Riemannian manifold and an open geodesic invex set with respect to . Assume that is differentiable GQBPIX with respect to and . Then is GQBIX with respect to and , where .*

*Proof. *Let and . Since is GQBPIX with respect to and , we have
Dividing the above inequality by and letting , we get
which shows that is GQBIX with respect to and . This completes the proof.

Theorem 24. *Let be a Riemannian manifold and an open geodesic invex set with respect to . Suppose that satisfies condition (C) and is continuous with respect to the second argument. If is differentiable GQBIX with respect to and , then is GQBPIX with respect to and , where for all and for some .*

*Proof. *Let . Since is a geodesic invex set with respect to , there exists exactly one geodesic such that
Let . Consider the set

In order to show that is GQBPIX, we have to show that . It is evident that is equivalent to the set
If , then, by the continuity of and , the set
is also nonempty. Hence, it is sufficient to show that , to complete the proof.

Suppose now that . We then have , for some , , and . By the definition of GQBIX of, it follows, considering the pair and , that
Similarly, considering the pair and , it follows that
Hence by condition (*C*), we have
Now (55), together with the fact that , imply that
Note that (56) holds for any . Now suppose that . Let and let . By the continuity of , , and , we can find such that for all , we have
Now, by the Mean Value Theorem, there exists such that
where . The left-hand side is positive by our hypothesis, but the right-hand size is zero by (56), as , and hence, we have a contradiction. This completes the proof.

We can also easily obtain the following results.

Theorem 25. *Let be a Riemannian manifold, let be an open geodesic invex set with respect to , and let be differentiable GPBPIX with respect to and . Then is GPBIX with respect to and , where .*

Theorem 26. *Let be a Riemannian manifold, let be an open geodesic invex set with respect to , and let be differentiable GPBPIX with respect to and . Then is GQBIX with respect to and , where .*

*Remark 27. *Theorems 25 and 26 generalize the known results from to Riemannian manifolds.

#### 4. Optimality Conditions and Duality

In this section, we discuss a multiobjective optimization problem (VOP) involving generalized GBIX functions and obtain the Kuhn-Tucker sufficient conditions for a feasible point of (VOP) to be an efficient or properly efficient solution. We also formulate a Mond-Weir type dual for (VOP) and give various types of duality results. All these conclusions extend the corresponding results on (see, e.g., [2, 4, 5, 7, 8] and the references therein) to Riemannian manifolds under the assumptions of GBIX functions and their generalization introduced in Section 2.

Let be a Riemannian manifold and let be an open invex set with respect to . We are concerned with the following multiple objective optimization problem: where and are differentiable functions. Let be the set of feasible solutions for (VOP), and .

For vector inequalities we adopt the usual notions. If , then

*Definition 28 (see [34]). *A feasible point is said to be an efficient solution of (VOP) if there exists no other feasible point such that .

*Definition 29 (see [34]). *The point is said to be properly efficient of (VOP) if it is efficient for (VOP) and if there exists a scalar such that, for each ,
for some such that , whenever is feasible for (VOP) and .

Theorem 30. *Let be a feasible solution to (VOP). Assume that for every feasible point , there exist scalars and , such that
**
Suppose that , are GBIX with respect to and at , and , is GBIX with respect to and at . If for any , then is a properly efficient solution for (VOP).*

*Proof. *Since , and , are GBIX, from condition (62), we get
Thus,
which implies that
where
Hence, from Theorem 4.11 of [35], is a properly efficient solution for (VOP). This completes the proof.

*Remark 31. *Theorem 30 is a generalization of Theorem 5.5 in [23].

Theorem 32. *Let be a feasible solution for (VOP). If there exist scalars , , , , , such that the triplet satisfies (62) in Theorem 30. Assume that is strictly GBIX with respect to and at , and , is GBIX with respect to and at . Then is an efficient solution for problem (VOP).*

*Proof. *Suppose that is not an efficient solution for (VOP). Then there exists a feasible point such that
Since is strictly GBIX, we conclude
Also, the GBIX of yields
Adding (68) and (69), we obtain a contradiction to (62). This completes the proof.

*Remark 33. *Proceeding along the same lines as in Theorem 30, it can be easily seen that becomes properly efficient to (VOP) in the above theorem, if , for all .

Theorem 34. *Suppose that there exist a feasible point and scalars , , , such that (62) of Theorem 30 holds. Let be GPBIX with respect to and at and let , be GQBIX with respect to and at . If and for any , then is a properly efficient solution for (VOP).*

*Proof. *Since , and , are GQBIX functions, we obtain
which along with (62) yields
Since is GPBIX, the above inequality implies that
Thus, we conclude that minimize , under the constraint , where
Therefore, is a properly efficient for (VOP). This completes the proof.

Theorem 35. *Let be a feasible point for (VOP). Assume that there exist scalars , , , , such that (62) of Theorem 30 holds. Let be strictly GPBIX with respect to and at , and let , be GQBIX with respect to and at . If for any , then is an efficient solution for problem (VOP).*

*Proof. *Suppose that is not efficient for (VOP). Then, there exists a feasible of (VOP) such that
which yields
It follows from the strict GPBIX of that
Also, from the GQBIX of , , we conclude
The proof now is similar to the proof of Theorem 32. This completes the proof.

*Remark 36. *Similarly as in Theorem 34, it can be easily seen that becomes properly efficient for (VOP) in the above theorem if for all .

*Remark 37. *In Theorems 34 and 35, the results still hold when is replaced by with .

We now consider the following Mond-Weir vector dual of (VOP): for all , and and are differentiable functions on .

We now prove various duality results for (VOP) and (MVD).

Theorem 38 (weak duality). *Let and be feasible for (VOP) and (MVD), respectively. If also either *(a)* for all and is GPBIX with respect to and at , is GQBIX with respect to and at , and , , or*(b)* is strictly GPBIX with respect to and at , is GQBIX with respect to and at , and ,**then .*

*Proof. *We proceed by contradiction. If , then for , and , we get
or for , , we have
Now since is GPBIX or strictly GPBIX, the above two inequalities both give
It follows from (79) that
and the GQBIX of gives
By adding (84) and (86), we obtain a contradiction to (78). This completes the proof.

Theorem 39 (strong duality). *Let be an efficient solution for (VOP) at which the Kuhn-Tucker conditions are satisfied. If for all feasible solutions of (MVD) is strictly GPBIX with respect to and and is GQBIX with respect to and , where for all , then there exists such that is efficient for (MVD) and the objective function values of (VOP) and (MVD) are equal.*

*Proof. *The assumption in the above theorem implies that there exist scalars , , , and , such that the Kuhn-Tucker conditions hold:
which gives that the triplet is feasible for (MVD). If is not efficient, then there exists a feasible for (MVD) such that , which contradicts the weak duality. This completes the proof.

Theorem 40. *Let be feasible for (VOP), and let be feasible for (MVD) such that
**
Suppose that is strictly GPBIX at with respect to and , and is GQBIX at with respect to and . If and , then .*

*Proof. *Let . Since is GQBIX and is feasible for (MVD), we conclude
It follows from (78) that
Again from the strict GPBIX of , we have
This is a contradiction. Therefore, . This completes the proof.

Theorem 41. *Suppose that there exist a feasible for (VOP) and a feasible for (MVD) such that
**
If for , and
**
is GPBIX/GQBIX with respect to and at , then is properly efficient for (VOP). Also if for each feasible of (MVD),
**
is GPBIX/GQBIX with respect to and and , , then is properly efficient of (MVD).*

*Proof. *Suppose that is not an efficient solution for (VOP). Then there exists a feasible for (VOP) such that
Using condition (92), we contradict the weak duality. Thus, is efficient for (VOP). If is not properly efficient for (VOP), then there exist a feasible and an index such that
for all and all such that whenever . Again utilizing condition (92), we obtain
for all and all such that
whenever
Since can be made large, hence for , we get the inequality
which contradicts the weak duality. Therefore, is properly efficient for (VOP).

To prove the second half of this theorem, let us assume on the contrary that is not an efficient solution of (MVD). Then there exists a feasible point of (MVD) such that
By applying condition (92), we get a contradiction to weak duality. Hence, is an efficient solution of (MVD).

If is not properly efficient for (MVD), then there exist a feasible of (MVD) and an index such that
for all and for all such that
whenever