#### Abstract

A new subclass of analytic functions is introduced. For this class, firstly the Fekete-Szegö type coefficient inequalities are derived. Various known or new special cases of our results are also pointed out. Secondly some applications of our main results involving the Owa-Srivastava fractional operator are considered. Thus, as one of these applications of our result, we obtain the Fekete-Szegö type inequality for a class of normalized analytic functions, which is defined here by means of the Hadamard product (or convolution) and the Owa-Srivastava fractional operator.

#### 1. Introduction and Definitions

Let denote the class of functions of the form which are analytic in the unit disk Also let denote the subclass of consisting of univalent functions in .

Fekete and Szegö [1] proved a noticeable result that the estimate holds for . The result is sharp in the sense that for each there is a function in the class under consideration for which equality holds.

The coefficient functional on represents various geometric quantities as well as in the sense that this behaves well with respect to the rotation; namely,

In fact, rather than the simplest case when we have several important ones. For example, represents , where denotes the Schwarzian derivative Moreover, the first two nontrivial coefficients of the th root transform of with the power series (1) are written by so that where

Thus it is quite natural to ask about inequalities for corresponding to subclasses of . This is called Fekete-Szegö problem. Actually, many authors have considered this problem for typical classes of univalent functions (see, e.g., [1–12]).

For two functions and , analytic in , we say that the function is subordinate to in , and we write if there exists a Schwarz function , analytic in , with such that In particular, if the function is univalent in , the above subordination is equivalent to

Let be an analytic function with which maps the open unit disk onto a star-like region with respect to and is symmetric with respect to the real axis.

This paper contains analogues of (3) for the following classes of analytic functions.

*Definition 1. *Let
A function is said to be in the class if it satisfies the following subordination condition:
where is defined to be the same as above for .

*Remark 2. *(i) If we set
in Definition 1, then we have the class
which consists of functions satisfying
This class was introduced by Bansal [13].(ii)If we set

in Definition 1, then we have a new class
which consists of functions satisfying
Taking
in (25), we have the class
which consists of functions satisfying
This class was introduced by Bansal [14].(iii)If we set
in (25), then we have the class
which consists of functions satisfying
This class was introduced by Murugusundaramoorthy and Magesh [15]. The subclass was studied by MacGregor [16].

We denote by the class of the analytic functions in with

We will need the following lemmas.

Lemma 3 (see [12]). *If with , then
**
When or , equality holds true if and only if is or one of its rotations. If , then equality holds true if and only if is or one of its rotations. If , then the equality holds true if and only if
**
or one of its rotations. If , then the equality holds true if and only if is the reciprocal of one of the functions such that the equality holds true in the case when .**Although the above upper bound is sharp, in the case when , it can be further improved as follows:
*

Lemma 4 (see [17]). *Let with . Then for any complex number **
and the result is sharp for the functions given by
*

#### 2. Fekete-Szegö Problem for the Function Class

By making use of Lemma 3, we first prove the Fekete-Szegö type inequalities asserted by Theorem 5 below.

Theorem 5. *Let
**
Also let
**
where
**
If given by (1) belongs to the function class , then
**
where
**
If , then
**
Furthermore, if , then
**
Each of these results is sharp.*

* Proof. *Since , we have
where
From (46), we have
Since is univalent and , the function
is analytic and has a positive real part in . Also we have
Thus by (47) and (49) we get
Taking into account (50), we obtain
where
The assertion of Theorem 5 now follows by an application of Lemma 3. On the other hand, using (51) for the values of , we have
Similarly, for the values of , we get
To show that the bounds asserted by Theorem 5 are sharp, we define the following functions:
with
by
and the functions and , with
by
respectively. Then, clearly, the functions , , and . We also write
If or , then the equality in Theorem 5 holds true if and only if is or one of its rotations. When , then the equality holds true if and only if is or one of its rotations. If , then the equality holds true if and only if is or one of its rotations. If , then the equality holds true if and only if is or one of its rotations.

By making use of Lemma 4, we immediately obtain the following Fekete-Szegö type inequality.

Theorem 6. *Let
**
Also let
**
where
**
If given by (1) belongs to the function class , then for any complex number **
The result is sharp.*

*Remark 7. *The coefficient bounds for and are special cases of those asserted by Theorem 5.

Taking in Theorem 6, we have the following corollary.

Corollary 8 (see [13]). *Let
**
Also let
**
where
**
If given by (1) belongs to the function class , then for any complex number **
The result is sharp.*

If we set in Theorem 6, then we have So we get the following corollary.

Corollary 9. *Let
**
Also let
**
If given by (1) belongs to the function class , then for any complex number **
The result is sharp.*

Putting in Corollary 9, we obtain the following corollary.

Corollary 10 (see [13]). *Let
**
Also let
**
If given by (1) belongs to the function class , then for any complex number **
The result is sharp.*

Also putting , , , and in Corollary 9, we obtain the following corollary.

Corollary 11. *Let . If given by (1) belongs to the function class , then for any complex number *

#### 3. Applications to Analytic Functions Defined by Using Fractional Calculus Operators and Convolution

The subject of fractional calculus (i.e., calculus of integrals and derivatives of any arbitrary real or complex order) has gained considerable popularity and importance during the past three decades or so. For the applications of the results given in the preceding sections, we first introduce the class , which is defined by means of the Hadamard product (or convolution) and a certain operator of fractional calculus, known as the Owa-Srivastava operator (see, e.g., [18, 19]).

*Definition 12. *The fractional integral of order is defined, for a function , by
where the function is analytic in a simply connected domain of the complex -plane containing the origin, and the multiplicity of is removed by requiring to be real when .

*Definition 13. *The fractional derivative of order is defined, for a function , by
where is constrained, and the multiplicity of is removed, as in Definition 12.

*Definition 14. *Under the hypotheses of Definition 13, the fractional derivative of order is defined, for a function , by

Using Definitions 12, 13, and 14, fractional derivatives, and fractional integrals, Owa and Srivastava [20] introduced the operator defined by
This operator is known as the Owa-Srivastava operator. In terms of the Owa-Srivastava operator defined by (81), we now introduce the function class in the following way:
Note that the function class is a special case of the function class when
Now suppose that
Since
we can obtain the coefficient estimates for functions in the class from the corresponding estimates for functions in the class . By applying Theorem 5 to the following Hadamard product (or convolution):
we get the following theorem after an obvious change of the parameter .

Theorem 15. *Let
**
Also let
**
where
**
If given by (1) belongs to the function class , then
**
where
**
Each of these results is sharp.*

When corresponds to the Owa-Srivastava operator given in (82), we obtain For and given by (93) and (94), respectively, Theorem 15 reduces to the following result.

Theorem 16. *Let
**
Also let
**
where
**
If given by (1) belongs to the function class , then
**
where
**
Each of these results is sharp.*

#### Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.