#### Abstract

We study analytically and numerically the interaction potentials between a pair of quark and antiquark on D3, M2, and M5 branes. These potentials are obtained using Maldacena’s method involving Wilson loops and present confining and nonconfining behaviours in different situations that we explore in this work. In particular, at the near horizon geometry, the potentials are nonconfining in agreement with conformal field theory expectations. On the other side, far from horizon, the dual field theories are no longer conformal and the potentials present confinement. This is in agreement with the behaviour of strings in flat space where the string mimics the expected flux tube of QCD. A study of the transition between the confining/nonconfining regimes in the three different scenarios (D3, M2, and M5) is also performed.

#### 1. Introduction

Usually, in quantum field theory, the Wilson loop operator is defined aswhere denotes a closed loop in space-time and the trace is over the fundamental representation of the gauge field with symmetry. In the particular case of a rectangular loop (of sides and ), it is possible to calculate (in the limit ) the expectation value for the Wilson loop:where can be identified with the energy of the quark-antiquark pair in the static limit.

Soon after the conjecture about the duality between M/string theory in AdS spaces and conformal gauge field theories [1–5], Maldacena [6] and Rey and Yee [7] (MRY) proposed a method to calculate expectation values of the Wilson loop for the large limit of field theories. This limit is calculated from a string theory in a given background using the gauge/gravity duality.

In this method, the expectation value of the Wilson loop is related to the worldsheet area of a string whose boundary is the loop in question such that

Maldacena used this approach to calculate the quark-antiquark potential for the string in the background [6] obtaining a nonconfining potential for the infinitely massive quark-antiquark pair, consistent with the conformal symmetry of the dual super Yang-Mills theory. In other backgrounds, the quark-antiquark potential can be confining as shown, for instance, in [8], where a confinement criterion was obtained.

This approach can also be extended to the finite-temperature case [9, 10] by considering an AdS Schwarzschild background. In this case, the temperature of the conformal dual theory is identified with the Hawking temperature of the black hole [11]. This situation also leads to a nonconfining potential for the quark-antiquark interaction.

The thermodynamics of D-brane probes in a black hole background were treated in [12]. These systems are holographically dual to a small number of flavours in a finite-temperature gauge theory. First-order phase transitions were found characterised by a confinement/deconfinement transition of quarks.

A phenomenological approach was also considered calculating the Wilson loop for the string in some holographic AdS/QCD models. For instance, the hard-wall model exhibits a confining behaviour [13, 14] reproducing the Cornell potential. At finite temperature, this calculation gives a second-order phase transition describing qualitatively a confinement/deconfinement phase transition [15]. Then, it was shown that a Hawking-Page phase transition [16] should occur for the hard- and soft-wall models at finite temperature [17–21]. In particular, for the soft-wall model, an interesting estimate of the deconfinement temperature was found [17], compatible with QCD expectations.

In a recent paper, some geometric configurations of a static string on a D3 brane background [22] and also a string-like object on M2 and M5 brane backgrounds [23] were studied. These geometric configurations correspond to a gauge theory which describes the quark-antiquark interaction on the branes. For some specific geodesic regimes, we found confining interactions and for others nonconfining potentials were found.

In this paper, we perform a systematic analytical and numerical study of the quark-antiquark potentials in D3, M2, and M5 brane backgrounds analysing their confining/nonconfining behaviours in different situations, always at zero temperature. In particular, at the near horizon geometry, the potentials are nonconfining in agreement with conformal field theory expectations. On the other side, far from horizon, the dual field theories are no longer conformal and the potentials present confinement. This is in agreement with the expected behaviour of strings in flat space where the string mimics the flux tube model of QCD. In the cases of M2 and M5 branes in M-theory, we choose a cigar-shaped membrane object such that stringy picture of the dual flux tube also holds. We also focus in searching for the point in the geodesics at which the zero temperature confinement/deconfinement transition takes place.

#### 2. Wilson Loops in D3, M2, and M5 Brane Spaces

We start this study by considering the Wilson loop on the background generated by a large number of coincident D3 branes in string theory in 10-dimensional space-time. The Nambu-Goto string action [24]is employed on this background, where the scale was set to , are the coordinates of the string worldsheet, and is the background metric. The specific form of the metric is given in the next section. It is considered that the quark-antiquark pair is contained in the D3 brane world which is attached to the ends of the open string that lives in 10 dimensions. For simplicity, we work in a static string configuration that is represented in Figure 1. This is achieved considering the large quark masses limit. As in the original Maldacena’s proposal [6], it is necessary to consider heavy quarks in order to compute the expectation value of the Wilson loop. For this reason, actually the right hand side of (3) contains the contribution of these heavy quark masses so it diverges. As a consequence, in general, it is necessary to subtract the quark masses from the divergent integral of the potential in order to obtain a finite result.

The Wilson loop corresponds to a rectangle with sides and , where is some time interval. This rectangle is associated with the worldsheet surface as shown in Figure 2.

Thus the distance separation between the quark-antiquark pair may be computed starting from the geodesic of the static string on this D3 brane background. This distance turns out to be an expression in terms of and (resp., minimum and maximum for coordinate in the worldsheet, as shown in Figure 1. See [8, 22]):where

According to the MRY proposal the worldsheet area () is proportional to the energy interaction () between the quark-antiquark pair, so it may also be written down in terms of and :Actually, as we are going to show in the following sections, this expression for for the quark-antiquark potential is divergent in the limit where , meaning that the pair is located at the boundary of the D3 brane space. This happens because the integrand goes to 1 in that limit. Physically, this divergence can be interpreted as the inclusion of the quark masses in the potential . As we are considering the limit of large quark masses, this implies the divergence of . So, in order to obtain a finite interaction potential, we are going to subtract the quark masses from the expression of . Then, we write a renormalized quark-antiquark interaction potential as

We continue our study analysing the cases concerning M2 and M5 brane backgrounds. Since these backgrounds of 11-dimensional SUGRA correspond to M-theory objects, it is not possible to start from Nambu-Goto action. Instead we should start from a 11-dimensional membrane action in those backgrounds [24, 25]:where are world-volume indices with as the induced metric, are space-time indices with as the space-time metric, are the membrane coordinates, is a three-form field with strength , and sets the scale for the membrane (see [25]). After the compactification of one spatial dimension of the membrane wrapped along the 11th dimension of space-time, we are able to reduce the membrane in 11 dimensions to a string-like object in 10 dimensions (see [26]). As a result, we are able to work with string-like objects and similarly to the case of strings on D3 backgrounds, and we utilize the static configuration and the MRY proposal to get the distance separation and energy interaction between a pair of quark-antiquark on M2 (M5) branes (see [23]).

#### 3. D3 Brane

The solitonic solution of 10-dimensional supergravity that we are going to study is a space geometry generated by coincident D3 branes. This solution is usually written down as follows [2, 27]:where is a constant defined by .

Following the MRY approach, the calculation of the distance separation (5) and the static potential interaction (8) between a pair of quarks on the D3 brane was obtained in [22]:whereFollowing [8], we have that the quark mass must bewhich diverges in the limit .

In the following, we are going to study the distance separation (11) and the potential energy (12) of the quark-antiquark pair in various different situations in the D3 brane solution.

##### 3.1. Nonconfining Behaviour

Let us start our study considering the regime defined by which means that the quark is very massive. Also we take which means that we are in the near horizon geometry which corresponds approximately to the space. We take as the independent variable of parametrization with fixed . Then, the behaviour of the distance separation (11) against is analysed. The numerical result is shown in Figure 3(a), where we plot versus . This plot shows a monotonic decreasing behaviour of against .

**(a)**

**(b)**

The next step is to analyse the behaviour of the potential (12) against the separation (11). The numerical result is shown in Figure 3(b), where we plot versus . This plot shows an increasing function which goes to zero as increases. So one can conclude that this plot corresponds to a nonconfining potential which is essentially Coulomb-like, as the one found by Maldacena in [6] for the case of the pure AdS space. This result is also in agreement with [22] where a nonconfining potential was obtained in the regime with . The dual field theory in this case is the well-known SYM which is a superconformal field theory. Then the nonconfining behaviour found for the Wilson loop is in agreement with the conformal property of the dual theory.

##### 3.2. Confining Behaviour

Our next step is to analyse the regime (very massive quark) but with which corresponds to the region far from the horizon which is approximately a flat space geometry. First we perform a numerical study of the distance separation (11) against the minimum position of the string . The result of this analysis is presented in Figure 4(a), where we plot versus . This figure shows a monotonic increasing behaviour of against .

**(a)**

**(b)**

Then, the next step is to study the shape of the potential (12) against the separation distance (11). We did this numerical study and the result is presented in Figure 4(b), where we plot the behaviour of against .

Looking at Figure 4(b), we see an almost straight line with positive derivative indicating that this plot implies a confining potential. This result is in agreement with [22], where a linear confining potential was obtained in this regime for the quark-antiquark pair in D3 brane space. The dual theory in this case is no longer conformal, since we are far from the horizon. Here, we can understand this picture as a string in flat space which mimics the confining flux tube of QCD.

##### 3.3. Deconfinement/Confinement Transition

In previous sections, we obtained confining and nonconfining behaviours for the potential energy (12) against the separation distance (11) in D3 brane space for different regimes of compared with . So, we expect that a transition should occur between the regimes (far from the horizon) and (near the horizon).

In this section, we work with for values in the regime such that we may find some deconfinement/confinement transition. Note that this is not a thermal phase transition since we are working at zero temperature. Instead, the expected transition should be related to the geometry of the D3 brane space.

First, we present in Figure 5(a) a plot showing how varies against . This picture shows a minimum value of which we call .

**(a)**

**(b)**

Note that it is also possible to find from (11) and (13). Using these equations, we get an expression whose root is precisely :Note that the solutions for depend only on the cutoff . Ideally, we should let the ratio , in order to have the quark-antiquark pair at the boundary of the D3-brane space. However, since we are using a numerical method to evaluate , we should choose a convenient numerical cutoff . Some numerical solutions for this equation are shown in Table 1, considering some big cutoff values. So, from this table, it is clear the dependence of the minimum on the cutoff .

The potential interaction (12) against the separation distance (11) in this regime is presented in Figure 5(b). Note that, in this case, is not a function of in the usual sense. In this figure, we can notice that there are two branches: the inferior one is a nonconfining Coulomb-like potential, and the superior one is a confining potential that has a monotonic increasing behaviour as is increased.

From Figure 5(b), our analysis shows that the condition corresponds to nonconfining behaviour and condition corresponds to confining one. This is the expected transition in the confinement/deconfinement behaviour of the quark-antiquark pair potential in D3 brane space. The transition seems to occur near the region . It is important to remark that this is not a thermal phase transition since we are working at zero temperature and the transition is of geometrical nature.

#### 4. M2 Brane

In the previous section, we presented an analysis of the Wilson loop for the D3 brane background. Here in this section and in the following we are going to present a similar discussion for other backgrounds such as M2 and M5 brane spaces. Although these background spaces belong to 11-dimensional M-theory that must correspond to higher dimensional objects like membranes, it is possible to do a dimensional reduction. This consists in compactifying one dimension of the membrane along one spatial direction, in order to have a string-like configuration in 10-dimensional background spaces. For details, see [23, 26].

We start the study of confinement with the MRY method in SUGRA backgrounds with the case of the space generated by coincident M2 branes. The 11-dimensional supergravity M2 brane solution is given by the metric (see [4, 5, 25]):where is a constant defined by , is the number of coincident branes, is Plank’s length in eleven dimensions, and is the differential solid angle for seven-sphere.

In a previous work [23], the distance separation () and static potential () for a pair of quarks in a M2 brane space were obtained:where () is the minimum (maximum) value of coordinate associated with the string-like object obtained by dimensional reduction and . Again following [8], we can compute the quark mass as follows:which diverges if we let .

##### 4.1. Nonconfining Behaviour

In this section, we work in the geometric regime which corresponds to very massive quarks and with which means that we are in the near horizon geometry which is approximately . First, we plot in Figure 6(a), the distance (17) against . We can notice from this plot that has a monotonic decreasing behaviour as is increased.

**(a)**

**(b)**

Next, we plot in Figure 6(b) the potential interaction (18) against the distance of the quark-antiquark pair (17). As we can note from this plot, the potential interaction in this case turns out to have a Coulomb-like nonconfining behaviour. This is in agreement with the result obtained in this same regime in [23] and with the fact that the dual field theory is conformal, since we are in the near horizon geometry which is approximately .

##### 4.2. Confining Behaviour

Here, we still work in the regime , but with , which corresponds to the region far from the horizon which is approximately a flat space. Now we plot in Figure 7(a) the rationalised distance (17) against . From this plot, we can see that the distance has an increasing behaviour as is increased. Notice that this behaviour is almost linear.

**(a)**

**(b)**

Next, continuing in the same geometric regime, we plot in Figure 7(b) the potential (18) against (17). We can see from this plot that potential interaction has positive derivative, which means a confining behaviour. This behaviour is expected since we are working in the region far from the horizon of the M2 brane geometry where the dual field theory is nonconformal.

##### 4.3. Deconfinement/Confinement Transition

In the last subsections, we had a nonconfining behaviour at the regime and a confining one at . So in this section we look for a transition behaviour at a middle term regime .

First we plot the distance between quarks (17) against , which is shown in Figure 8(a). From this plot we notice that there is a minimum at . Also we can get this value as a root of an equation that can be derived from (17):

**(a)**

**(b)**

Some solutions of this equation are presented on Table 2. Note that here the solutions for only depend on the cutoff . Ideally, we should let , but since we are doing a numerical analysis we need to fix to some big but finite value. So in this table we show these solutions for some big cutoff values.

Next we plot in Figure 8(b) the potential (18) against the distance (17). Note that in this case, the potential is no longer a function in the usual sense of the separation . We notice from this plot that there are two branches: the inferior one corresponding to a nonconfining Coulomb-like potential and the superior one corresponding to a confining potential. Also, from our analysis of the last plots, we can conclude that for the potential is a nonconfining one while for the potential is a confining one.

#### 5. M5 Brane

Now we analyse the confinement behaviour of a quark-antiquark pair using the MRY method in the 11-dimensional SUGRA background space generated by coincident M5 branes. The metric solution is [4, 5]:where is a constant given by .

According to [23], the distance separation and the potential interaction of a pair of quarks are given bywhere () is the minimum (maximum) value of coordinate of the string-like object obtained from dimensional reduction and . Following [8] we can compute the quark mass:which is divergent in the limit .

##### 5.1. Nonconfining Behaviour

We work here in the regime which means that the quarks are very massive and with corresponding the region near horizon which in this case is approximately an geometry. For this regime the distance between the pair of quarks (22) against is plotted in Figure 9(a). This plot shows that the distance has a monotonic decreasing behaviour as is increased.

**(a)**

**(b)**

Next we plot in Figure 9(b) the potential interaction (23) against the distance separation between quarks (22). We can see from this plot that the potential shows a nonconfining behaviour: it has a positive slope going asymptotically to zero as increases. This is the expected behaviour since we are in the near horizon region where the metric is approximately an compatible with a conformal field theory.

##### 5.2. Confining Behaviour

In this subsection, we still work in the regime but with (far from the horizon) which corresponds to an approximately flat space geometry. In this regime, we plot in Figure 10(a) the distance separation (22) against . We can see from this plot that shows an almost linear behaviour as is increasing.

**(a)**

**(b)**

Next we plot in Figure 10(b) the potential interaction between the pair of quarks (23) against the distance between quarks (22). From this plot we can notice that the potential shows a confining behaviour: it has a positive slope as is increased. This behaviour is expected since we are working in the region far from the brane which approaches asymptotically a flat space so that the dual field theory is no longer conformal.

##### 5.3. Deconfinement/Confinement Transition

In the last subsections, we found nonconfining potential behaviour at and confining potential behaviour at . In this section, we work in the regime and look for a confinement/deconfinement transition. First we plot in Figure 11(a) the distance separation between quarks (22) against . From this plot we can notice that there is a minimum at the position . We can also get as a root of equation that is obtained deriving (22):Some solutions of this equation are shown in Table 3. Note that the solutions for depend only on the cutoff . Ideally, we should let , but since we are doing a numerical analysis we need to fix the ratio . So, in this table, we show these solutions considering some big cutoff values.

**(a)**

**(b)**

Finally we plot in Figure 11(b) the potential interaction (23) against the distance separation between quarks (22). Note that, in this case, the potential is not a function, in the usual sense, of the separation . From this plot, we can see that we have two branches: the superior one corresponds to a confining potential interaction, since we can observe that it has a positive derivative as increases. On the other side, the inferior one corresponds to a nonconfining potential.

Also we can conclude from these plots that values with correspond to a nonconfining behaviour and values with correspond to a confining behaviour.

#### 6. Discussions and Conclusion

We have analysed the Wilson loops for D3, M2, and M5 brane backgrounds using the MRY approach. As was discussed previously in [22, 23], these backgrounds imply confining and nonconfining quark-antiquark potentials depending on the geometric regime considered. We investigated here these situations further and mainly the transition between these two confinement behaviours.

Note that in [22] the authors have discussed only the case of the D3 brane space from the analytical point of view. Here, we presented an analytical/numerical approach, extended to the cases of the M2 and M5 brane spaces. Furthermore, in [22, 23], the confinement/deconfinement transition was not discussed at all.

In general, for the three geometries studied, we notice that as the distance separation has a monotonic decreasing behaviour with , one finds a nonconfining potential interaction. This situation occurs at the regimes and , which corresponds to heavy quark masses in the asymptotic AdS geometries ( assumes the values , , and for the geometries D3, M2, and M5 branes, resp.).

On the other hand, when the distance separation is a monotonic increasing function of , one finds a confining potential interaction. This situation occurs at the regimes and , which corresponds to heavy quark masses in flat space geometries. This confining behaviour can be understood looking at the metric in the region far from the brane. In this case the metric approaches a flat space-time so that the dual field theory is no longer conformal. This situation is analogous to a string in flat space which mimics the flux tube model of QCD showing confinement.

We found out that the confinement/deconfinement transition occurs at a point in the regime for the D3, M2, and M5 brane backgrounds. The point is where the nonmonotonic distance function of is a minimum. The value of depends on and and we have tabulated possible values in Tables 1, 2, and 3, for each geometry depending on the chosen cutoff. This situation occurs at the regime (heavy quark) and corresponds to a transition between the AdS and flat space geometries. All these situations were analysed at zero temperature, so that the nature of the transitions is purely geometrical and not thermodynamical. It is important to remark that the confinement/deconfinement transition shows up here as a nonanalyticity of the potential in terms of the separation of the quark-antiquark pair. This is evident in Figures 5(b), 8(b), and 11(b).

It would be interesting to analyse if this discussion can be extended to other Wilson loop configurations where corrections are present [28–31].

#### Conflicts of Interest

The authors declare that they have no conflicts of interest.

#### Acknowledgments

The authors would like to thank C. Hoyos for interesting discussions at the* Strings at Dunes* Conference in Natal, Brazil, 2016, where a previous version of this work was presented. They would like also to acknowledge CNPq, Brazilian agency, for financial support.