#### Abstract

The sensitivity of the atmospheric neutrino experiments to the nonstandard flavor-dependent interaction in neutrino propagation is studied under the assumption that only nonvanishing components of the nonstandard matter effect are the electron and tau neutrino components , and , and that the tau-tau component satisfies the constraint which is suggested from the high energy behavior for atmospheric neutrino data. It is shown that the Super-Kamiokande (SK) data for 4438 days constrains at 2.5 (98.8%) CL whereas the future Hyper-Kamiokande experiment for the same period of time as SK will constrain as at 2.5CL from the energy rate analysis and the energy spectrum analysis will give even tighter bounds on and .

#### 1. Introduction

From the experiments with solar, atmospheric, reactor, and accelerator neutrinos it is now established that neutrinos have masses and mixing [1]. Neutrino oscillations in the standard three-flavor scheme are described by three mixing angles, , , and , one CP phase , and two independent mass-squared differences, and . The sets of the parameters (, ) and (, ) were determined by the solar neutrino experiments and the KamLAND experiment and by atmospheric and long baseline neutrino experiments, respectively. was determined by the reactor experiments and the long baseline experiments [1]. The only oscillation parameters which are still undetermined are the value of the CP phase and the sign of (the mass hierarchy). In the future neutrino long-baseline experiments with intense neutrino beams the signs of and are expected to be determined [2, 3]. As in the case of B factories [4, 5], such high precision measurements will enable us to search for deviation from the standard three-flavor oscillations (see, e.g., [6]). Among such possibilities, in this paper, we will discuss the effective nonstandard neutral current flavor-dependent neutrino interaction with matter [7–9], given by where and stand for fermions (the only relevant ones are electrons, and quarks), is the Fermi coupling constant, and stands for a projection operator that is either or . If the interaction (1) exists, then the standard matter effect [7, 10] is modified. We will discuss atmospheric neutrinos which go through the Earth, so we make an approximation that the number densities of electrons (), protons, and neutrons are equal (this assumption is not valid in other environments, e.g., in the Sun.). Defining , the Hermitian matrix of the matter potential becomes where stands for the matter effect due to the charged current interaction in the standard case. With this matter potential, the Dirac equation for neutrinos in matter becomes where is the leptonic mixing matrix defined by and , , and .

Constraints on have been discussed by many authors, from atmospheric neutrinos [11–15], from colliders [16], from the compilation of various neutrino data [17], from solar neutrinos [18–20], from or scatterings [21, 22], from solar and reactor neutrinos [23], and from solar, reactor, and accelerator neutrinos [24]. Since the coefficients in (2) are given by , considering the constraints in these references, we have the following limits [25] at 90%CL: From (5) we observe that the bounds on , , and are much weaker than those on .

On the other hand, the nonstandard interaction (NSI) with components must be consistent with the high-energy atmospheric neutrino data. It was pointed out in [26, 27] that the relation should hold for the matter potential (2) to be consistent with the high-energy atmospheric neutrino data, which suggest the behavior of the disappearance oscillation probability where and are the oscillation parameters in the two-flavor formalism. In [28] it was shown that, in the high-energy behavior of the disappearance oscillation probability in the presence of the matter potential (2), and imply and .

Taking into account the various constraints described above, in the present paper we take the ansatz and analyze the sensitivity to the parameters of the atmospheric neutrino experiment at Super-Kamiokande and the future Hyper-Kamiokande (HK) facility [29] (as far as is concerned, the ansatz (9) is believed to be the best fit of the high energy atmospheric neutrino data at present. So as long as the true value of satisfies the relation (6), even if we analyze the data assuming that is a free parameter, the allowed region in (, ) and the sensitivity to NSI are not expected to change very much, because the region of , which does not satisfy (6), gets an additional contribution of and is not supposed to contribute to enlarge the allowed region or to increase the sensitivity to NSI).

The constraints on and from the atmospheric neutrino have been discussed in [30] along with those from the long-baseline experiments, in [31] by the Super-Kamiokande Collaboration, in [32–34] on the future extension of the IceCube experiment, and in [35] in the global analysis, with the ansatz different from ours.

The sensitivity of the ongoing accelerator experiments to the nonstandard interaction in propagation was studied by various authors. The constraints have been obtained from the MINOS experiment in [36], from the MINOS data using the same ansatz as the present paper in [37, 38], from the MINOS data from the viewpoint of degeneracy of and NSI in [39], from appearance in MINOS and T2K in [40], from the OPERA experiment in [41, 42], and from the LHC experiment in [43]. As for the future long-baseline experiments, the sensitivity of the INO experiment was discussed in [44], that of the reactor and superbeam experiments was discussed in [45], that of the T2KK experiment was studied in [28, 46], and that of the LBNE experiment was discussed in [43, 47]. The sensitivity of neutrino factories [6] was studied in various contexts: the sensitivity to NSI [48–50], the confusion with the effect of [51], the optimization [52], resolving degeneracy with two baselines [53, 54], and the relation with nonunitary mixing [55].

The paper is organized as follows. In Section 2, we analyze the SK atmospheric neutrino data and give the constraints on the parameters from the SK atmospheric neutrino data. In Section 3, we discuss the sensitivity to of the future Hyper-Kamiokande atmospheric neutrino experiment. In Section 4, we draw our conclusions.

#### 2. The Constraint of the Super-Kamiokande Atmospheric Neutrino Experiment on and

In this section we discuss the constraint of the SK atmospheric neutrino experiment on the nonstandard interaction in propagation with the ansatz (9). The independent degrees of freedom in addition to those in the standard oscillation scenario are , and .

The SK atmospheric neutrino data we analyze here is those in [56] for 4438 days. In [56], the contained events, the partially contained events, and the upward going events are divided into a few categories. Since we have been unable to reproduce all their results of the Monte Carlo simulation, we have combined the two sub-GeV -like data set in one, the two multi-GeV -like in one, the two partially contained event data set and the multi-GeV -like in one, and the three upward going in one. Reference [56] gives information on the ten zenith angle bins, while that on the energy bins is not given, so we perform analysis with the ten zenith angle bins and one energy bin; that is, we perform the rate analysis as far as the energy is concerned.

The analysis was performed with the codes which were used in [57–59]. is defined as In (10) for the sub-GeV, multi-GeV, and upward going events are defined by The summation on runs over the ten zenith angle bins for each , ; ) stands for the neutrino and antineutrino data of the numbers of the sub-GeV, multi-GeV, and upward going events, () stands for the theoretical prediction for the number of -like events () which is produced from () that originates from () through the oscillation process (), and it is expressed as the product of the oscillation probability (), the flux (), the cross section, the number of the targets, and the detection efficiency. stands for the uncertainty in the overall flux normalization for the sub-GeV, multi-GeV, and upward going events and () stands for the uncertainty in the relative normalization between flux ( flux) for the sub-GeV () and multi-GeV () events, respectively. It is understood that is minimized with respect to , (), , (), and . We have put the systematic errors and we have assumed that and for the contained events are free parameters as in [60]. We have omitted the other uncertainties, like the spectral index, the relative normalization between PC and FC and up-down correlation, and so forth, for simplicity. In (10) the sum of each is optimized with respect to the mixing angle , the mass squared difference , the Dirac CP phase , and the phase of the parameter . The other oscillation parameters give little effect on , so we have fixed them as , , and .

The result for the Super-Kamiokande data for 4438 days is given in Figure 1. The best-fit point for the normal (inverted) hierarchy is (, ) = (−1.0, 0.0) ((3.0, 1.7)) and the value of at this point is 79.0 (78.6) for 50 degrees of freedom, and goodness of fit is 2.8 (2.7) CL, respectively. The best-fit point is different from the standard case (, ) = (0, 0), and this may be not only because we have been unable to reproduce the Monte Carlo simulation by the Super-Kamiokande group, but also because we use only the information on the energy rate and the sensitivity to NSI is lost due to the destructive phenomena between the lower and higher energy bins (see the discussions in Section 3.1). The difference of the value of for the standard case and that for the best-fit point for the normal (inverted) hierarchy is (3.4) for 2 degrees of freedom (1.1CL (1.3CL)), respectively, and the standard case is certainly acceptable for the both mass hierarchies in our analysis. From Figure 1 we can read off the allowed region for , and we conclude that the allowed region for is approximately

**(a) SK 4438 days NH**

**(b) SK 4438 days IH**

#### 3. Sensitivity of the Hyper-Kamiokande Atmospheric Neutrino Experiment to and

In this section we discuss the potential sensitivity of HK to and . Here we assume for simplicity that the the Hyper-Kamiokande detector has the same detection efficiencies as those of SK and that the fiducial volume of HK is twenty times as large as that of SK. Since HK is a future experiment, the simulated numbers of events are used as “the experimental data,” and we vary and as well as the standard oscillation parameters trying to fit to “the experimental data.” Here we perform an analysis on the assumption that we know the mass hierarchy, because some hint on the mass hierarchy is expected to be available at some confidence level by the time HK will accumulate the atmospheric neutrino data for twenty years.

Since “the experimental data” are the simulated numbers of events, we can perform a energy spectrum analysis, assuming that the detection efficiency and so forth are all equal among neutrinos and antineutrinos. Before we study the sensitivity to NSI, as a benchmark of our analysis, we have investigated the significance of the wrong mass hierarchy with our code, assuming the standard oscillation scenario and using different numbers of the energy bins. By comparing our result with the one in [29], we have found that our analysis on the mass hierarchy gives a result similar to that in [29], when we work with two energy bins in the contained events (the sub-GeV and multi-GeV events) and the systematic errors which are slightly different from those in [29] (when we have more than two energy bins, our results would lead to too large significance of the wrong mass hierarchy in the case of the standard oscillation scenario, compared with the one in [29]. Also in the presence of NSI, our study with more than two energy bins would give allowed regions which are smaller than those by the two-energy-bin analysis. So we will take two energy bins in the energy spectrum analysis to be conservative throughout this paper). We have checked that the sensitivity to NSI is not affected significantly by changing the systematic errors. As for the upward going events, since our ansatz (9) is taken in such a way that the oscillation probability with approaches to the one with the standard scenario in the high energy limit, the upward going events are expected to give a small contribution to the significance of NSI. So in the case of the energy spectrum analysis we will work with two energy bins in the contained events and a single energy bin in the upward going events.

##### 3.1. The Case with the Standard Oscillation Scenario

First of all, let us discuss the case where “the experimental data” is the one obtained with the standard oscillation scenario. The values of the oscillation parameters which are used to obtain “the experimental data” are the following (to distinguish the oscillation parameters for the “the experimental data” (; ; ; , etc.) and those for the numbers of events (; ; ; , etc.) for fitting, the parameters with a bar denote those for “the experimental data,” whereas those without a bar denote the parameters for the numbers of events for fitting):As in the case of the analysis of the SK data, we vary the oscillation parameters , , , and while fixing the other oscillation parameters , , and .

In the energy rate analysis, is the same as (10) where the numbers of events are calculated with the standard oscillation scenario with , , and given by (14), and we have assumed that all the systematic errors except are the same as those in (12) in the analysis of SK data. , which is the uncertainty in the relative normalization between the flux, was chosen because this value was used in the energy spectrum analysis on the significance of the wrong mass hierarchy to give the result close to that in [29] (see the discussions below).

In the spectrum analysis, on the other hand, and are replaced byIn (16) we have introduced the relative normalization, which in general depends on the flavor and the energy of the events, between the upward and downward going bins: and is the index which separates the upward and downward bins. The indices and stand for the lower () and higher () energy bins, and the threshold energy is chosen so that the numbers of events for the lower and higher energy bins are approximately equal, and in the case of the sub-GeV events, GeV, and in the case of the multi-GeV events, the threshold energy is GeV, respectively, for all the zenith angle bins. We have put the systematic errors as follows: All the systematic errors in (18) except and are the same as those in (12) in Section 2 and those used in [60]. is the uncertainty in the relative normalization between the multi-GeV flux and it was 0.05 in (12). is the uncertainty in the relative normalization between the upward and downward going multi-GeV -like events and it was 0.008 in the analysis of SK data [60]. The choice of these systematic errors (18) and (19) and the index has been made so that the result of our analysis on the mass hierarchy is close to that in [29], and we have checked that the choice of (18) and (19) and the index do not affect the sensitivity to NSI significantly. Notice that we have included the systematic uncertainty for the up-down correlation, unlike our analysis of the Super-Kamiokande data in Section 2. We have omitted the other systematic uncertainties, such as the spectral index, for simplicity. The systematic error for the spectral index affects all the numbers of events for each energy universally, and it is not expected to affect the sensitivity to NSI very much.

The results from the energy rate (spectrum) analysis are given by the upper (lower) panel in Figure 2. From the energy rate analysis we have at 2.5CL. On the other hand, from the energy spectrum analysis we get and at 2.5 (98.8%) CL for the normal hierarchy and to and at 2.5 (98.8%) CL for the inverted hierarchy.

From Figure 2 we note two things. Firstly, the allowed regions from the energy spectrum analysis (the lower panel) are much smaller than those from the energy rate analysis (the upper panel) for both mass hierarchies. Secondly, the allowed regions (the right panel) for the inverted hierarchy are wider than those (the left panel) for the normal hierarchy for both rate and spectrum analyses.

To understand these phenomena, we have plotted in Figure 3 for the multi-GeV events, which are expected to be sensitive to the matter effect and therefore to , as a function of in the case of . In plotting the figures in Figure 3, we have taken into account only the statistical errors for simplicity, and we assume that the HK detector could distinguish neutrinos and antineutrinos for both -like and -like events in all the energy ranges of the multi-GeV events and that the detection efficiency is the same for both neutrinos and antineutrinos. Since the SK collaboration distinguish neutrinos and antineutrinos only for the multi-GeV -like events [56], our assumption here may not be realistic, and the separate plots for neutrinos or for antineutrinos except for the -like events should be regarded as information for theoretical consideration. The two figures ((a) and (b)) in the top row are the results of the energy rate analysis. The two figures ((c) and (d)) in the middle row are the results of the energy spectrum analysis with two energy bins for the separate neutrino or antineutrino events. The two figures ((e) and (f)) in the bottom row are the results of the energy spectrum analysis with two energy bins of neutrinos and antineutrinos combined. Comparing the figures ((a) and (b)) in the top row and those ((e) and (f)) in the bottom row, we see that, even if some of the data set in the spectrum analysis have a sensitivity to the effect of , the data in the rate analysis does not necessarily have a sensitivity to particularly for , for both mass hierarchies. While it is not clear to us why the sensitivity is lost only for , we have found that, if we try to fit the same data with the numbers of events with the wrong mass hierarchy, then the plot becomes left-right reversed, that is, the sensitivity is lost only for . On the other hand, by comparing the figures ((c) and (d)) in the middle row and those ((e) and (f)) in the bottom row, we see that, in the case of the inverted mass hierarchy, even though the separate data has a sensitivity to , the combined data loses a sensitivity. We could not explain these phenomena using the analytic expression for the oscillation probability, but we interpret this loss of sensitivity as a destructive phenomenon between neutrinos and antineutrinos in the rate analysis and between the lower and higher energy bins in the spectrum analysis for the inverted mass hierarchy.

**(a) Rate NH**

**(b) Rate IH**

**(c) 2bin NH**

**(d) 2bin IH**

**(e) 2bin NH combined**

**(f) 2bin IH combined**

To visualize how this destructive phenomenon happens in terms of the numbers of events, we have plotted in Figure 4 the difference of the numbers of the multi-GeV -like events with standard scenario and those with NSI for a typical case: From Figure 4 we see that each positive and negative contribution to the difference in the events (Figure 4(a)) is almost cancelled by negative and positive contribution in the events in Figure 4(b), so significance is reduced in the combined events (Figure 4(c)).

**(a)**multi-GeV IH

**(b)**multi-GeV IH

**(c)**multi-GeV IHAlthough we have not thoroughly investigated, according to our investigation for a specific case (), this destructive phenomenon does not happen for ; that is, distinction between neutrinos and antineutrinos does not make much difference on the sensitivity to . This conclusion is consistent with the result of [44], in which the sensitivity to was studied, although they took a set of assumptions different from ours. This destructive phenomenon seems to be characteristic to the sensitivity to because of the asymmetry between the cases for and for .

It is expected that the HK experiment will be able to use information on the energy spectrum, so we believe that the allowed region in the lower panel in Figure 2 with the energy spectrum analysis reflects the true HK sensitivity more than that in the upper panel does.

##### 3.2. The Case in the Presence of NSI

Next let us discuss the case where “the experimental data” is the one obtained with . The analysis is the same as the one in Section 3.1, except that the “the experimental data” is produced assuming the presence of NSI, and here we perform only an energy spectrum analysis with two energy bins. The results are shown in Figure 5, where the allowed regions at 2.5CL ( for 2 degrees of freedom) around the true points are depicted. The straight lines in Figure 5 stand for the approximate bound from the SK atmospheric neutrinos in Figure 1, and we have examined only the points below these straight lines. As seen from Figure 5, the errors in and are small for in the case of the normal hierarchy and for in the case of the inverted hierarchy. The errors are large otherwise, and the reason that the errors are large is because a sensitivity is lost due to a destructive phenomenon between neutrinos and antineutrinos as was discussed in Section 3.1.

**(a) NH**

**(b) IH**

We note in passing that there are a couple of points in Figure 5, where the allowed region has an additional isolated island. This is regarded as so-called parameter degeneracy, which is classified into the intrinsic degeneracy [61], the sign degeneracy [62], and the octant degeneracy [63, 64] in the standard three-flavor framework, in the presence of the NSI. Since little is known about parameter degeneracy in the presence of the new physics and since the study of the subject is beyond the scope of this paper, we do not discuss parameter degeneracy here.

#### 4. Conclusions

In this paper we have studied the constraint of the SK atmospheric neutrino data on the nonstandard flavor-dependent interaction in neutrino propagation with the ansatz (9). From the SK atmospheric neutrino data for 4438 days, we have obtained the bound at 2.5CL, while we have little constraint on .

We have also discussed the sensitivity of the future HK atmospheric neutrino experiment to NSI by analyses with the energy rate and with the energy spectrum. If nature is described by the standard oscillation scenario, then the HK atmospheric neutrino data will give us the bound at 2.5CL from the energy rate analysis, and from the energy spectrum analysis it will restrict to and at 2.5 (98.8%) CL for the normal hierarchy and to and at 2.5 (98.8%) CL for the inverted hierarchy. On the other hand, if nature is described by NSI with the ansatz (9), then HK will measure the NSI parameters and relatively well for in the case of the normal hierarchy and for in the case of the inverted hierarchy.

We have shown that it is important to use information on the energy spectrum to obtain strong constraint, because a sensitivity to NSI would be lost due to destructive phenomena between the low and high energy events. If there is a way to distinguish between neutrinos and antineutrinos, as is done by the SK collaboration [56] for the -like multi-GeV events, also for the multi-GeV -like events, then the sensitivity to NSI would be greatly improved, because in this case we can avoid destructive phenomena between neutrinos and antineutrinos.

Finally let us discuss some prospects for the global analysis with these future atmospheric results and the solar neutrino results. In [35] the global analysis was performed with all the data presently available, and the conclusion was that the constraints from the solar and KamLAND data are stronger than those from the atmospheric and long baseline experiments. Furthermore, because of the slight difference between the best fit values for the solar and KamLAND data in the standard scenario, their result may suggest a nonzero value for the NSI parameter , which is a function of (), (), and . From our results in Section 3, if the mass hierarchy is normal, then we see that the errors of the parameters and obtained in the future Hyper-Kamiokande experiment may be comparable to or even smaller than the present error of . So HK may be able to contribute to give further constraints on the parameter, although more detailed study will be required to be conclusive.

While HK is expected to play an important role in measurement of in the standard three-flavor scenario using the JPARC beam, HK has also a potential for new physics with atmospheric neutrinos. Search for NSI may lead to physics beyond the Standard Model, and the effects of NSI at HK deserves further studies.

#### Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

#### Acknowledgment

This research was partly supported by a Grant-in-Aid for Scientific Research of the Ministry of Education, Science and Culture, under Grants no. 24540281 and no. 25105009.