#### Abstract

In the present work, by making use of Gegenbauer polynomials, we introduce and study a certain family of -pseudo bistarlike and -pseudo biconvex functions with respect to symmetrical points defined in the open unit disk. We obtain estimates for initial coefficients and solve the Fekete–Szeg problem for functions that belong to this family. Furthermore, we give connections to some of the earlier known results.

#### 1. Introduction

In [1], Legendre studied orthogonal polynomials comprehensively. The importance of orthogonal polynomials for contemporary mathematics as well as for a wide range of their applications in physics and engineering is beyond any doubt. It is well known that these polynomials play an essential role in problems of approximation theory. They occur in the theory of differential and integral equations as well as in mathematical statistics. Their applications in quantum mechanics, scattering theory, automatic control, signal analysis, and axially symmetric potential theory are also known [2, 3]. In practice, Gegenbauer polynomials are a special case of orthogonal polynomials. They are representatively related to typically real functions as discovered in [4]. Typically, real functions play an important role in geometric function theory because of the relation and its role in estimating coefficient bounds, where indicates the family of univalent functions in the unit disk with real coefficients and denotes the closed convex hull of .

On this subject in geometric function theory, so-called Fekete–Szegö type inequalities (or problems) estimate some upper bounds for for holomorphic univalent functions. Its origin was in the disproof by Fekete and Szego of the 1933 conjecture of Littlewood and Paley that the coefficients of odd univalent functions are bounded by unity (see [5])

We consider , the set of functions , which are holomorphic in the open unit disk , having the following form:

Let be denoted as the subfamily of consisting of the functions which are univalent in .

According to the Koebe one-quarter theorem [6], each function has an inverse , which fulfillsandwhere

We say that a function is biunivalent in if both and are univalent in . Let indicate the class of biunivalent functions in given by (1). Starting with Srivastava et al. pioneering work [7] on the subject, a large number of works related to the subject have been presented (see, for example [4, 8–24]). We notice that the family is not empty. Some examples of functions in the class arewith the corresponding inverse functionsrespectively. We recall here other common examples of functions that are not members of , namely,

So far, the coefficient estimate problem for each of the following Taylor–Maclaurin coefficients:for functions is still an open problem.

We say that a function is starlike with respect to symmetrical points if (see [25])

The subset of all such functions is denoted by .

The family of starlike functions with respect to symmetrical points obviously includes the family of convex functions with respect to symmetrical points , satisfying the following condition:

We say that a function is -pseudostarlike function in if (see [26])

We consider two functions and that are holomorphic in . We say that the function is subordinate to if there exists a Schwarz function holomorphic in with and such that . This subordination is denoted by or . It is well known that (see [27]) if the function is univalent in , then if and only if and .

Recently, Amourah et al. [28] have studied Gegenbauer polynomials , which are given by the following recurrence relation.

For a nonzero real constant , a generating function of Gegenbauer polynomials is defined bywhere and . For fixed , the function is holomorphic in , so it can be expanded in a Taylor series, and note that if , where , thenwhere is the Gegenbauer polynomial of the degree .

Obviously, generates nothing when . Thus, the generating function of the Gegenbauer polynomial is set to be

Furthermore, it is worth to mention that a normalization of to be greater than is desirable [3, 29]. Gegenbauer polynomials can also be defined by the following recurrence relations:

The initial values are expressed as

*Remark 1. *By choosing the particular values of , the Gegenbauer polynomial reduces to well-known polynomials. These special cases are as follows:(1)Taking , we obtain Chebyshev polynomials(2)Taking , we obtain Legendre polynomials

#### 2. Main Results

This section starts with defining the new family .

*Definition 1. *We consider that , , and that is a nonzero real constant. A function is said to be in the family if both and its inverse given by (4) fulfill the following subordinations:In particular, if we choose and in Definition 1, the family becomes the family , which Wanas defined in [30].

If we choose in Definition 1, the family reduces to the family , which Wanas gave in [31]. As a consequence, we will generalize, in the following theorem. his main result given in [31, Theorem 2.1]

Theorem 1. *Let f, a holomorphic function, given by (1). If is in the family , where , , , and , then the following are the upper bonds for the Taylor–Maclaurin coefficients and :*

*Proof. *We suppose that is an element of the family , and by Definition 1, and its inverse are biunivalent functions of from to , satisfying subordinations (17) and (18), and hence, we have two holomorphic functions:andWe haveSince the holomorphic functions and are from to with , and for , and hence, we have the well-known factBy comparing corresponding coefficients in (22) and (23), we obtainIt follows from (25) and (27) thatBy adding (26) to (28), we obtainBy eliminating from (30) and (31), we obtainUsing (16), (24) and (32), we reach the first inequalityNext, by subtracting (28) from (26), we obtainThus, in view of (29) and (30), (34) givesFinally, by applying (16), we complete the proof byIn particular, Theorem 1 gives the following corollaries for some specific values of , and .

If , then the family , the class of -pseudo bistarlike functions with respect to symmetrical points.

Corollary 1. *For , , and is a nonzero real constant, let be in the family . Then,**And we obtain*

If , then the family , the class of -pseudo biconvex functions with respect to symmetrical points.

Corollary 2. *For , , and is a nonzero real constant, let be in the family . Then,**And we obtain*

If , Theorem 1 gives the following corollary by Wanas [31] for the well-known family .

Corollary 3 (see [31]). *For , , and , let be in the family . Then,*

And we obtain

If , Theorem 1 the following corollary by Wanas [30] for the well-known family .

Corollary 4 (see [30]). *For and , let be in the family . Then,*

And we obtain

Furthermore, if , Corollary 2 gives the following corollary by Wanas and Majeed [32] for the well-known family .

Corollary 5 (see [32]). *For , let be in the family . Then,*

And we obtain

In the next theorem, we present the “Fekete–Szegö problem” for the family .

Theorem 2. *For , , and , , and is a nonzero real constant, let be in the family . Then,*

*Proof. *In the light of (32) and (34), we deduce thatwhereAccording to (16), we deduce thatAfter some computations, we obtainIn particular, putting in Theorem 2, we conclude the following result.

Corollary 6. *For , , and , and is a nonzero real constant, let be in the family . Then,*

#### 3. Conclusion

In the present work, we obtain a family of -pseudo bistarlike and -pseudo biconvex functions with respect to symmetrical points defined by Gegenbauer polynomials. We generated Taylor–Maclaurin coefficient inequalities of functions belonging to this family and viewed the famous Fekete–Szegö problem. Furthermore, by specifying parameters, consequences of this family are mentioned.

#### Data Availability

The data are available upon request.

#### Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.