#### Abstract

We study the flavor changing neutral current couplings of new heavy quarks through the models at the LHC. We calculate the cross sections for the signal and the corresponding standard model background processes. Considering the present limits on the mass of new heavy quarks and the boson, we performed an analysis to investigate the parameter space (mixing and mass) through different models. For FCNC mixing parameter and the mass GeV and new heavy quark mass GeV at the LHC with TeV, we find the cross section for single production of new heavy quarks associated with top quarks as fb, fb, fb, and fb within the , , , and models, respectively. It is shown that the sensitivity would benefit from the flavor tagging.

#### 1. Introduction

Addition of new heavy quarks would require the extension of the flavor mixing in charged current interactions as well as the extension of Higgs sector in the standard model (SM). A large number of new heavy quark pairs can be produced through their color charges at the Large Hadron Collider (LHC). However, due to the expected smallness of the mixing between the new heavy quarks and known quarks through charged current interactions, the production and decay modes can be affected by the flavor changing neutral current (FCNC) interactions. A new symmetry beyond the SM is expected to explain the smallness of the mixing. We may anticipate the new physics discovery by observing large anomalous couplings in the heavy quark sector. The couplings of the new heavy quarks can be enhanced to observable levels within some new physics models. In numerous phenomenological studies (see [1] and references therein), a lot of extensions of the SM foresee the extra gauge bosons, the boson in particular. Flavor changing neutral currents can be induced by an extra gauge boson . The boson in the models using an extra group can have tree-level or an effective (where and both can be the up-type quarks or down-type quarks) couplings. The , , and models corresponding to the specific values of the mixing angle in the model with different couplings to the fermions and the leptophobic model with the couplings to quarks but no couplings to leptons are among the special names of the models [2].

The ATLAS and CMS collaborations have performed extensive searches of new vector resonances at the LHC. We summarize briefly these searches that exploited data from the run at TeV and TeV, as well as the corresponding constraints on boson masses. The most stringent limits come from searches with leptonic final states (): GeV [3] and GeV [4], GeV [5] (more recently GeV [3]) for the boson predicted by the extensions, also extending to the mass limit of GeV [3, 6] for a gauge boson with sequential couplings. The results from ATLAS experiment exclude a leptophobic decaying to with a mass less than GeV at CL [7], while the CMS experiment excludes a top-color decaying to with a mass less than GeV at CL [8]. These searches assume rather narrow width for the boson (). From the electroweak precision data analysis, the improved lower limits on the mass are given in the range 1100–1500 GeV, which gives a limit on the – mixing about [2]. The limits on the boson mass favors higher center of mass energy collisions for direct observation of the signal. Using dilepton searches with LHC data, the dark matter constraints have been analyzed in [9, 10] in the regime .

A work performed in [11, 12] presents the effects of FCNC interactions induced by an additional boson on the single top quark and top quark pair production at the LHC ( TeV). The relevant signal cross sections have been calculated and particularly the benefit from flavor tagging to identify the signal has been discussed. Considering an existence of sizeable couplings to the new heavy quarks, the boson decay width and branchings, as well as the production rates, can be quite different from the expectations of usual search scenarios.

In the models of interest new heavy quarks can have some mixing with the SM quarks. For example, in composite Higgs model [13] the lightest new heavy quark couples predominantly to the heavier SM quarks (top and bottom quarks). In the models of vector-like quarks (VLQ) [14] they are expected to couple preferentially to third-generation quarks and they can have flavor changing neutral current couplings, in addition to the charged current decays characteristic of chiral quarks. Within the model the isosinglet quarks [15] are predicted and they can decay to the quarks of the SM. The new heavy quarks can be produced dominantly in pairs through strong interactions for masses around 1 TeV in the collisions of the LHC with a center of mass energy of TeV. The single production of new heavy quarks would only be dominant over pair production for the large quark masses [16], it is model dependent, and it could be suppressed if the mixing with SM quarks is small.

There are searches for pair production and single production of new heavy quarks at the LHC. The ATLAS and CMS collaborations focused on decay modes of new heavy quarks into a massive vector boson and a third-generation quark assuming a branching ratio, based on of collision data at TeV, and set lower mass limit for up-type new heavy quark as GeV [17] and GeV [18].

In this work, we investigate the single production of new heavy quarks via FCNC interactions through boson exchange at the LHC. This paper aims at studying the signal and background in detail within the same MC framework, and the relevant interaction vertices are implemented into the MC software. Analyzing the signal observability (via contour plots) for different mass values of the boson and new heavy quarks as well as the mixing parameter through FCNC interactions are another feature of the work. In Section 2, we calculate the decay widths and branching ratios of boson for the mass range 1500–3000 GeV in the framework of different models. An analysis of the parameter space of mass and coupling strength is given for the single production of new heavy quarks at the LHC in Section 3. We analyzed the signal observability for the FCNC interactions. We consider both and single new heavy quark productions for the purpose of enriching the signal statistics even at the small couplings. For the quark decay we consider mode within the interested parameter space. The analysis for the signal significance is given in Section 4 and the work ends up with the conclusions as given in Section 5.

#### 2. FCNC Interactions

In the gauge eigenstate basis, we can write the additional neutral current Lagrangian related to the gauge symmetry by following the formalism given in [11, 19, 20] where are the chiral couplings of boson with fermions and . is the gauge coupling of and Here, it is presumed that there is no mixing between the and bosons as favored by the precision data. If the chiral couplings are nondiagonal matrices flavor changing neutral currents (FCNCs) will arise. FCNC couplings come out by fermion mixing if the couplings are diagonal but nonuniversal. Here, we assume that all FCNCs are in the left-hand sector and only for up-type quarks, and the specific form of the mixing matrix is used. Our assumptions can be considered within a model framework (e.g., a particular class of string models [21, 22]) and an ad hoc illustration of a constrained formalism. In the interaction basis the FCNC for the up-type quarks can be given by where the chiral couplings can be written as

The effects of these FCNCs may always arise in the sectors, both up-type and down-type ones after diagonalizing their mass matrices. For simplicity, we suppose that the neutral current couplings to for the right-handed up-sector and down-sector are family universal and flavor diagonal in the interaction basis. In this case, unitary rotations () can maintain the right-handed couplings flavor diagonal, and left-handed sector becomes nondiagonal. The chiral couplings of in the fermion mass eigenstate basis are given by Here, the matrix can be written as ; due to our supposition that the down-sector has no mixing it becomes . The flavor mixing in the left-handed quark fields is simply relevant to this . One can find the couplings with the parametrization (where the generation index runs from 1 to 3) for the matrix by assuming the up-sector diagonalization and using unitarity of the CKM matrix.

The FCNC effects from the exchange have been studied for the down-type sector and implications in flavor physics through -meson decays [23–28] and -meson mixing [20, 29–33]. These effects have also been studied for up-type quark sector in top quark production [11, 34–38]. The parameters for different models are listed in Table 1. In numerical calculations, the coupling is taken as for the models.

In our model, the chiral couplings can be written as

The values of the matrix elements , , and are used as given in [39] by taking into account and . For a comparison, we also calculate the cross sections using the scenario of equal parameters (where runs from to ).

For the FCNC constraints from mixing with parameter , we follow the calculations performed in [11] and find that the contribution from boson (through FCNC effects) can be obtained as , where . With the given parametrizations above, this is translated into the result that as long as the combination is less than about the experimental bounds can be well satisfied in different models that we have studied in this work.

#### 3. Single Production of New Heavy Quarks

For numerical calculations we have implemented the interaction vertices into the CalcHEP program package [40]. The decay widths of boson for different mass values within different models are tabulated in Table 2. In consideration of the parameter , the couplings to both the left-handed and right-handed ones become universal and family diagonal. In this case, it is hard to see the FCNC effects on the decay widths and cross sections. For the FCNC effects on the decay width, we take the parameter as shown in Figure 1. All these scenarios of models predict a narrow decay width ranging from to for depending on the mass of boson foreseen by different models, for the considered set of parameters. The effect of the FCNC reduces the decay width in the relevant mass range. The decay widths are compared with similar results from [11, 12] for to prove the implementation. Unless otherwise stated throughout this work, we use the FCNC mixing parameter and the mass value of quark GeV and the mass value of new heavy charged lepton GeV and new heavy neutrino GeV. The branching ratios of boson decays depending on boson mass predicted by different models are given in Figures 2–9; specifically they are given in Figures 2, 4, and 6 for the diagonal couplings to quarks and leptons, while they are given in Figures 3, 5, and 7 for the FCNC couplings to different flavors of up-sector quarks within the , , and models, respectively. In Figures 8 and 9, the branchings for leptophobic boson decays to pair of quarks with diagonal couplings and FCNC couplings are presented depending on boson mass.

The cross sections obtained by using parton distribution function library CTEQ6L [41] for the process depending on the boson mass at the LHC (13 TeV) are given in Figures 10 and 11. Here, the boson contributes through the - and -channel diagrams, and the cross sections of associated production of single top quarks and single new heavy quarks ( and ) in the final state are summed. For this process the cross section at TeV is almost 8 times as large as the case at TeV.

We plot the distributions of the -quark in the signal process with GeV for the parameter at the center of mass energy of 13 TeV as shown in Figure 12. A high cut decreases the background importantly without affecting much the signal cross section in the related mass range. The rapidity distribution of the bottom quarks ( and ) from the signal are shown in Figure 13 at the collision energy of 13 TeV. In order to enhance the statistics we sum up the and distributions. There is a bump in the -quark rapidity distribution with the extending tails to . For the analysis, the suitable cuts are GeV, , and GeV. The cut is also useful well above the top quark mass. We also apply invariant mass cut (for system) to make analysis with the signal and background.

The signal cross sections () in the invariant mass interval of , for the process , are given in Tables 3, 4, 5, and 6 for different values of FCNC parameter (ranging from 0.01 to 0.5) within the () model, where new heavy quark masses are taken 600 GeV, 700 GeV, and 800 GeV. For the () model and the FCNC parameter for , , , and , the cross sections () are presented in Tables 7, 8, 9, and 10, respectively. The cross sections for the corresponding background are given in Table 11 for the chosen invariant mass interval.

Figures 14, 15, 16, and 17 show the invariant mass distribution of the system for the signal (with and sGeV) of different models and background at the LHC with TeV.

#### 4. Analysis

For the analysis, two types of backgrounds are considered. One has the same final state () as expected for the signal processes and the other (pair productions of top quarks are both related to -jets) is the irreducible background and contributes to similar final state. The ratio of the cross sections for pair production of top quarks at the TeV and TeV is about . The ratio of the cross sections for process is found to be about for considered models. It is expected that an improvement in the statistical significance (for the center of mass energy TeV when compared to the case of TeV) will be obtained. For the analysis, a high transverse momentum () cut for the -jets and the other jets can be applied. The results, employing the variable cuts ( GeV) for different mass values and the rapidity cuts () for the central detector coverage, are given in Tables 12, 13, 14, and 15, where the numbers of signal () and background () events are calculated by taking into consideration integrated luminosity of per year. For the FCNC coupling parameter , the LHC is able to measure the mass up to about GeV with the associated productions of the new heavy quark and top quark. The statistical significance () values for the final state are given in Tables 12–15 for different boson masses.

In the analysis, we reconstruct the invariant mass of system around the boson mass shown in Figures 14, 15, 16, and 17. We assume top quark decay in the form of , where the boson can decay leptonically or hadronically. In the final state , we assume the -tagging efficiency as for each of the -quarks. We take into account the channel in which one of the bosons decays leptonically, while the other decays hadronically. We calculate the cross section of the background in the mass bin widths for each value; as an example of the model, for GeV we take the invariant mass interval GeV, and we find the background cross section pb for process .

We plot the observability contours in the plane of model parameters , for different models as shown in Figure 18 at the LHC with TeV and . The curves with labels , , , and show the accessible regions (below the curves) of the model parameters at the LHC. For the model, the FCNC parameter bounds from 0.6–0.4 can be searched for the mass range of 1500–3000 GeV.

#### 5. Conclusion

We consider the associated productions of new heavy quark and top quark (with the subsequent decay channel ) through the exchange diagrams at the LHC. We find the discovery regions of the parameter space for the single productions of new heavy quarks through FCNC interactions with the new boson. In the models considered in this paper, the single production of new heavy quarks at the LHC can have the contributions from the couplings of and the FCNC couplings of (where ). For the FCNC parameter range ( means maximal to minimal FCNC) the LHC can have the potential to produce new heavy quarks which couple to the boson predicted by specific models. For a discussion on the decays, in this study we only consider . It may be possible that which does not have a CKM suppression like could compete with this decay mode depending on the mass difference of new and quarks. However, these channels require a new study which we will tackle as a separate topic of paper at another study.

#### Competing Interests

The authors declare that they have no competing interests.

#### Acknowledgments

O. Çakır’s work is supported in part by the Turkish Atomic Energy Authority (TAEK) under the project Grant no. 2016TAEK (CERN) A5.H6.F2-14.