Testing Normal Means: The Reconcilability of the 𝑃 Value and the Bayesian Evidence

Abstract

The problem of reconciling the frequentist and Bayesian evidence in testing statistical hypotheses has been extensively studied in the literature. Most of the existing work considers cases without the nuisance parameters which is not the frequently encountered situation since the presence of the nuisance parameters is very common in practice. In this paper, we consider the reconcilability of the Bayesian evidence against the null hypothesis 𝐻₀ in terms of the posterior probability of 𝐻₀ being true and the frequentist evidence against 𝐻₀ in terms of the 𝑃 value in testing normal means where the nuisance parameters are present. The reconcilability of evidence can be obtained both for testing a normal mean and for the Behrens-Fisher problem.

1. Introduction

In the problem of testing a statistical hypothesis 𝐻₀, a frequentist may give evidence against 𝐻₀ by the observed significance level, the 𝑃 value, while a Bayesian may give it by the posterior probability that 𝐻₀ is true. Lindley [1] illustrated the possible discrepancy between the Bayesian and the frequentist evidence. The relationship of these two measures of evidence is then extensively studied in the literature. Pratt [2] revealed that the 𝑃 values are usually approximately equal to the posterior probabilities in the one-sided testing problems. Casella and Berger [3] considered testing the one-sided hypothesis for a location parameter and showed that the lower bounds of the posterior probability over some reasonable classes of priors are exactly equal to the corresponding 𝑃 values in many cases. Some important papers which deal with the reconcilability of the Bayesian and frequentist evidence are Bartlett [4], Cox [5], Shafer [6], Berger and Delampady [7], and Berger and Sellke [8].

Although many researches have been carried out to deal with the problem of reconciling the Bayesian and frequentist evidence and some of them show that evidence is reconcilable in several specific situations, most of the existing work assumes that no other unknown parameters are present except the parameters of interest. In fact, we may be confronted with the nuisance parameters in various situations. In the location-scale settings, for example, when the location parameter is unknown, so is the scale parameter, in general.

However, in significance testing of hypotheses with the nuisance parameters, the classical 𝑃 values are typically not available. Tsui and Weerahandi [9], considering testing the one-sided hypothesis of the form 𝐻₀∶𝜃≤𝑐versus𝐻₁∶𝜃>𝑐, (1) where 𝜃 is the parameter of interest and 𝑐 is a fixed constant, introduced the concept of the generalized 𝑃 value, which appears to be useful in situations where conventional frequentist approaches do not provide useful solutions.

Tsui and Weerahandi [9] and some later relevant works formulated the generalized 𝑃 values for many specific examples. Hannig et al. [10] provided a general method for constructing the generalized 𝑃 value via fiducial inference.

In this paper, for the one-sided testing situations about normal means where the nuisance parameters are present, we study the reconcilability of the Bayesian evidence and the generalized 𝑃 value. It is shown that, under the conjugate class of prior distributions, the Bayesian evidence and the generalized 𝑃 value are reconcilable both for the problem of testing a normal mean and for the Behrens-Fisher problem.

This paper is organized as follows. In Section 2, we give the main results of the reconcilability of the 𝑃 value and the Bayesian evidence in testing normal means. Some conclusions and discussions are given in Section 3.

2. Main Results

In this section, we consider two testing problems in which the nuisance parameters are present. When no efficient classical frequentist evidence is available because of the presence of the nuisance parameters, we formulate the frequentist evidence by the generalized 𝑃 value.

2.1. One-Sample Normal Mean

Let 𝑋₁,…,𝑋_𝑛 be a random sample from a normal population 𝑁(𝜇,𝜎²), where both the mean 𝜇 and the variance 𝜎² are unknown. Consider now the following problem of testing the mean of a normal distribution 𝐻₀∶𝜇≤𝑐versus𝐻₁∶𝜇>𝑐, (2) where 𝑐 is a fixed constant.

For this testing problem, where the nuisance parameter is present, we can still obtain the classical 𝑃 value as 𝑝(𝑥)=𝑃𝑇_𝑛−1≤√𝑛(𝑐−𝑥)𝑠, (3) where 𝑇_𝑛−1 is a 𝑡-variable with 𝑛−1 degrees of freedom and 𝑥 and 𝑠² stand for the observed sample mean and sample variance, respectively.

To derive the Bayesian evidence, we need a prior for the parameters. One reasonable and conventional class of priors for 𝜇 and 𝜎² is the following conjugate class of prior distributions 𝐺_𝑐1: 𝜇∣𝜎²∼𝑁𝜇₀,𝜎²𝜅₀,1𝜎²∼Gamma𝜈₀2,𝜈₀𝜎²₀2, (4) where the prior parameters (𝜇₀,𝜅₀) can be interpreted as the mean and sample size of the normal prior observations and (𝜎²₀,𝜈₀) the sample variance and sample size of the Gamma prior observations.

Under (4) we have 𝜇∣𝑥,𝜎²∼𝑁𝜇_𝑛(𝑥),𝜎²𝜅_𝑛,1𝜎²∣𝑥∼Gamma𝜈_𝑛2,𝜈_𝑛𝜎²_𝑛(𝑥)2, (5) where 𝑥=(𝑥₁,…,𝑥_𝑛)^′,  𝜅_𝑛=𝜅₀+𝑛,  𝜇_𝑛(𝑥)=(𝜅₀𝜇₀+𝑛𝑥)/𝜅_𝑛, 𝜈_𝑛=𝜈₀+𝑛, and 𝜎²_𝑛(𝑥)=[𝜈₀𝜎²₀+(𝑛−1)𝑠²+𝜅₀𝑛(𝑥−𝜇₀)²/𝜅_𝑛]/𝜈_𝑛. Therefore, we can give the posterior density for (𝜇,𝜎²) as 𝜋𝜇,𝜎²=√∣𝑥𝜅_𝑛√−𝜅2𝜋𝜎exp^𝑛𝜇−𝜇_𝑛(𝑥)²2×𝜈_𝑛/2𝜎²_𝑛(𝑥)^𝜈𝑛/2Γ𝜈_𝑛/2𝜎^𝜈𝑛+2−𝜈exp^𝑛𝜎²_𝑛(𝑥)2𝜎²=√𝜅_𝑛𝜈_𝑛/2𝜎²_𝑛(𝑥)^𝜈𝑛/2√2𝜋Γ𝜈_𝑛/2𝜎^𝜈𝑛+3−𝜅×exp^𝑛𝜇−𝜇_𝑛(𝑥)²+𝜈_𝑛𝜎²_𝑛(𝑥)2𝜎². (6) Then the marginal posterior density for 𝜇 can be obtained by integrating out 𝜎² as 𝜋=√𝜇∣𝑥𝜅_𝑛𝜈_𝑛/2𝜎²_𝑛(𝑥)^−1/2Γ𝜈_𝑛+1/2Γ𝜈_𝑛×𝜅/21+^𝑛𝜇−𝜇_𝑛(𝑥)²𝜈_𝑛𝜎²_𝑛(𝑥)^{−(𝜈𝑛+1)/2}, (7) from which we know that √𝜅_𝑛𝜇−𝜇_𝑛(𝑥)𝜎_𝑛(𝑥)∼𝑡𝜈_𝑛. (8) Consequently, the posterior probability of 𝐻₀ being true is 𝑃𝐻₀∣𝑥=𝑃𝑇_𝜈𝑛≤𝜅_𝑛𝜎²_𝑛(𝑥)𝑐−𝜇_𝑛(𝑥), (9) where 𝑇_𝜈𝑛 is a 𝑡-variable with 𝜈_𝑛 degrees of freedom. Notice that if 𝜇₀=𝑐, we have lim_{𝜅0,𝜎0→0}𝑃𝐻₀∣𝑥=𝑃𝑇_𝜈𝑛≤𝜈_𝑛√𝑛−1𝑛(𝑐−𝑥)𝑠. (10)

Lemma 1. Let 𝑇 be a 𝑡-variable with 𝛼 degrees of freedom, where 𝛼 is a positive real number. Then for 𝑅=𝑇/√𝛼 and a fixed constant 𝑟, 𝑃(𝑅≤𝑟) is nonincreasing in 𝛼 if 𝑟≤0 and is nondecreasing in 𝛼 if 𝑟>0.

Proof. Suppose that 𝑋 is a nonpositive random variable obtained by the negative part of 𝑇; that is, the density of 𝑋 is 𝑓()=))𝑥,𝛼2Γ((𝛼+1/2√()𝑥𝛼𝜋Γ𝛼/21+²𝛼^{−(𝛼+1)/2},𝑥≤0. (11) Then the density of 𝑌=𝑋/√𝛼 is 𝑝=))𝑦,𝛼2Γ((𝛼+1/2√𝜋Γ(𝛼/2)1+𝑦^{2−(𝛼+1)/2},𝑦≤0. (12) By Theorem 3.3.2 in Lehmann [11], for any fixed nonpositive constant 𝑟, we have that 𝑃(𝑌≤𝑟) is nonincreasing in 𝛼 since it can be verified that the family of densities 𝑝(𝑦,𝛼) has monotone likelihood ratio in 𝑦. This implies that Lemma 1 holds for the case when 𝑟≤0 since 𝑃(𝑅≤𝑟)=𝑃(𝑋≤𝑟)/2. Since when 𝑟>0, we have 𝑃(𝑅≤𝑟)=1/2+𝑃(0≤𝑅≤𝑟), the proof for the latter case is completely analogous if we introduce a nonnegative random variable obtained by the positive part of 𝑇.

Now take 𝑟=√𝑛(𝑐−𝑥)/(√𝑛−1𝑠). By Lemma 1, for 𝑟≤0, we have 𝑃𝑇_𝜈𝑛√𝜈_𝑛𝑇≤𝑟≤𝑃^𝑛−1√𝑇𝑛−1≤𝑟=𝑃_𝑛−1≤√𝑛(𝑐−𝑥)𝑠. (13) Then comparing (3) and (10), for 𝜇₀=𝑐 and any fixed nonnegative 𝜈₀, we have lim_{𝜅0,𝜎0→0}𝑃𝐻₀∣𝑥<𝑝(𝑥),as𝑝(𝑥)<12, (14) which implies that inf_{𝜋∈𝐺𝑐1}𝑃𝐻₀∣𝑥<𝑝(𝑥),as𝑝(𝑥)<12. (15) The reconcilability of the Bayesian and frequentist evidence is therefore obtained in this testing problem. We summarize this as the following theorem.

Theorem 2. For testing the hypothesis of the form (2) under a normal distribution 𝑁(𝜇,𝜎²) with 𝜎² unknown, the Bayesian and frequentist lines of evidence are reconcilable under the conjugate class of priors (4).

2.2. Behrens-Fisher Problem

Now we turn to consider the Behrens-Fisher problem. It is a classical testing situation in which the nuisance parameters are present and no useful pivotal quantities are available. Suppose that 𝑋₁,…,𝑋_𝑚 and 𝑌₁,…,𝑌_𝑛 are two independent random samples from two normal populations 𝑁(𝜇₁,𝜎²₁) and 𝑁(𝜇₂,𝜎²₂), respectively, where both 𝜎²₁ and 𝜎²₂ are completely unspecified. We are interested in testing the hypothesis of the form 𝐻₀∶𝜇₁−𝜇₂≤𝑐versus𝐻₁∶𝜇₁−𝜇₂>𝑐, (16) where 𝑐 is a fixed constant.

In situations where the traditional frequentist approaches fail to provide useful solutions, the conception of the generalized 𝑃 values introduced by Tsui and Weerahandi [9] appears to be helpful in deriving the frequentist evidence for testing a statistical hypothesis. For this specific problem of testing hypothesis (16), we can give the generalized 𝑃 value as 𝑝(𝑥)=𝑃𝑥−√𝑦−𝑚−1𝑠₁𝑈√𝑚𝜒²_𝑚−1−√𝑛−1𝑠₂𝑉√𝑛𝜒²_𝑛−1𝑠≤𝑐=𝑃²√𝑛𝑇_𝑛−1−𝑠₁√𝑚𝑇_𝑚−1≤𝑐−𝑥−𝑦, (17) where 𝑈∼𝑁(0,1), 𝑉∼𝑁(0,1), 𝜒²_𝑚−1∼𝜒²(𝑚−1), 𝜒²_𝑛−1∼𝜒²(𝑛−1), 𝑇_𝑚−1∼𝑡(𝑚−1), and 𝑇_𝑛−1∼𝑡(𝑛−1).

In this problem, we consider the reconcilability of evidence under the following conjugate class of prior distributions 𝐺_𝑐2: 𝜇₁∣𝜎²₁∼𝑁𝜇₀₁,𝜎²₀₁𝜅₀₁,1𝜎²₁∼Gamma𝜈₀₁2,𝜈₀₁𝜎²₀₁2;𝜇₂∣𝜎²₂∼𝑁𝜇₀₂,𝜎²₀₂𝜅₀₂,1𝜎²₂∼Gamma𝜈₀₂2,𝜈₀₂𝜎²₀₂2. (18) Under 𝐺_𝑐2, the posterior density of (𝜇₁,𝜇₂,𝜎²₁,𝜎²₂) is 𝜋𝜇₁,𝜇₂,𝜎²₁,𝜎²₂=𝑓𝜇∣𝑥,𝑦₁,𝜇₂,𝜎²₁,𝜎²₂,𝑥,𝑦∫^∞_−∞∫^∞_−∞∫^∞₀∫^∞₀𝑓𝜇₁,𝜇₂,𝜎²₁,𝜎²₂,𝑥,𝑦𝑑𝜎²₁𝑑𝜎²₂𝑑𝜇₁𝑑𝜇₂, (19) where 𝑓𝜇₁,𝜇₂,𝜎²₁,𝜎²₂=−Σ,𝑥,𝑦exp^𝑚𝑖=1𝑥_𝑖−𝜇₁²+𝜅₀₁𝜇₁−𝜇₀₁²+𝜈₀₁𝜎²₀₁2𝜎²₁−Σ^𝑛_𝑖=1𝑦_𝑖−𝜇₂²+𝜅₀₂𝜇₂−𝜇₀₂²+𝜈₀₂𝜎²₀₂2𝜎²₂×𝜎²₁^{(𝑚+𝜈01+4)/2}𝜎²₂^{(𝑛+𝜈02+4)/2−1}. (20) Let 𝜃=𝜇₁−𝜇₂. Then the posterior density of (𝜃,𝜇₂,𝜎²₁,𝜎²₂) is 𝜋𝜃,𝜇₂,𝜎²₁,𝜎²₂=𝑓∣𝑥,𝑦𝜃,𝜇₂,𝜎²₁,𝜎²₂,𝑥,𝑦∫^∞_−∞∫^∞_−∞∫^∞₀∫^∞₀𝑓𝜇₁,𝜇₂,𝜎²₁,𝜎²₂,𝑥,𝑦𝑑𝜎²₁𝑑𝜎²₂𝑑𝜇₁𝑑𝜇₂, (21) where 𝑓𝜃,𝜇₂,𝜎²₁,𝜎²₂=−Σ,𝑥,𝑦exp^𝑚𝑖=1𝑥_𝑖−𝜃−𝜇₂²+𝜅₀₁𝜃+𝜇₂−𝜇₀₁²+𝜈₀₁𝜎²₀₁2𝜎²₁−Σ^𝑛_𝑖=1𝑦_𝑖−𝜇₂²+𝜅₀₂𝜇₂−𝜇₀₂²+𝜈₀₂𝜎²₀₂2𝜎²₂×𝜎²₁^{(𝑚+𝜈01+4)/2}𝜎²₂^{(𝑛+𝜈02+4)/2−1}. (22) So that the posterior probability of 𝐻₀ is 𝑃𝐻₀=∫∣𝑥_𝑐−∞∫^∞_−∞∫^∞₀∫^∞₀𝑓𝜃,𝜇₂,𝜎²₁,𝜎²₂,𝑥,𝑦𝑑𝜎²₁𝑑𝜎²₂𝑑𝜇₂𝑑𝜃∫^∞_−∞∫^∞_−∞∫^∞₀∫^∞₀𝑓𝜇₁,𝜇₂,𝜎²₁,𝜎²₂,𝑥,𝑦𝑑𝜎²₁𝑑𝜎²₂𝑑𝜇₁𝑑𝜇₂. (23) It is straightforward to check that lim_{𝜅01,𝜅02,𝜎01,𝜎02→0}𝑃𝐻₀=Γ∣𝑥𝑚+𝜈₀₁+2/2Γ𝑛+𝜈₀₂+2/2𝜋(𝑚−1)(𝑛−1)Γ𝑚+𝜈₀₁+1/2Γ𝑛+𝜈₀₂×+1/2_{∞−∞∞√𝑚(𝑥−𝑦−𝑐)/𝑠1+√𝑚𝑠2𝑟/𝑠1}𝑡1+²𝑚−1^{(−𝑚−𝜈01−2)/2}×𝑟1+²𝑛−1^{(−𝑛−𝜈02−2)/2}𝑑𝑡𝑑𝑟,=𝑃𝑥−𝑦−𝑚+𝜈₀₁+1𝑠₁𝑈𝑚+𝜈₀₂+2𝜒²_{𝑚+𝜈01+1}−𝑛+𝜈₀₂+1𝑠₂𝑉𝑛+𝜈₀₂+2𝜒²_{𝑚+𝜈01+1}𝑠≤𝑐=𝑃²𝑛+𝜈₀₂𝑇+2^{𝑛+𝜈02+1}−𝑠₁𝑚+𝜈₀₁𝑇+2^{𝑚+𝜈01+1}≤𝑐−𝑥−𝑦, (24) where 𝑥 and 𝑦 are the observation of the sample mean 𝑋 and 𝑌, respectively, 𝑠²₁ and 𝑠²₂ are that of the sample variance 𝑆²₁ and 𝑆²₂ respectively, 𝑈∼𝑁(0,1), 𝑉∼𝑁(0,1), 𝜒²_{𝑚+𝜈01+1}∼𝜒²(𝑚+𝜈₀₁+1) and 𝜒²_{𝑛+𝜈02+1}∼𝜒²(𝑛+𝜈₀₂+1).

Now we prove an interesting result that, when 𝑚 and 𝑛 are sufficiently large, the frequentist and Bayesian lines of evidence given respectively by (17) and (23) are reconcilable for any fixed 𝜈₀₁ and 𝜈₀₂ under the prior class of 𝐺_𝑐2.

Theorem 3. As min{𝑚,𝑛}→+∞, for any fixed 0<𝜈₀₁,𝜈₀₂<∞, we have lim_{𝜅01,𝜅02,𝜎01,𝜎02→0}𝑃𝐻₀(𝑥)(𝑥)<1∣𝑥<𝑝,𝑎𝑠𝑝2, (25) which implies that inf_{𝜋∈𝐺𝑐2}𝑃𝐻₀(𝑥)(𝑥)<1∣𝑥<𝑝,𝑎𝑠𝑝2. (26)

Proof. Let 𝐴₁=(𝑛−1)𝑠²₂𝑛𝜒²_𝑛−1,𝐵₁=𝑛+𝜈₀₂+1𝑠²₂𝑛+𝜈₀₂+2𝜒²_{𝑛+𝜈02+1}. (27)
(I) We first prove that, given 𝑠₂, as 𝑛 is sufficiently large, we have 𝐵<₁^𝑑𝐴₁. (28) In fact, for any 𝛾>0, as 𝑛 is sufficiently large, 𝑃𝐵₁<𝛾=𝑃𝜒²_{𝑛+𝜈02+1}>𝑛+𝜈₀₂+1𝑠²₂𝑛+𝜈₀₂𝛾,𝜒+2=𝑃²_{𝑛+𝜈02+1}>(𝑛−1)𝑠²₂,𝑛𝛾+𝜖𝑛,𝛾 (29) where 𝜖(𝑛,𝛾)=(𝜈₀₂+2)𝑠²₂/[(𝑛+𝜈₀₂+2)𝑛𝛾]→0, as 𝑛→∞. On the other hand, we have 𝑃𝐴₁<𝛾=𝑃𝜒²_𝑛−1>(𝑛−1)𝑠²₂.𝑛𝛾 (30) Since 𝜒²_𝑛+𝜈02^𝑑₊₁>𝜒²_𝑛−1 holds for any 𝑛, it follows that, as 𝑛→∞, 𝑃𝐵₁<𝛾>𝑃𝐴₁<𝛾. (31) That is, 𝐵<₁^𝑑𝐴₁. (32)
Similarly, as 𝑚→+∞, we have 𝐵₂∶=𝑚+𝜈₀₁+1𝑠²₁𝑚+𝜈₀₁+2𝜒²_𝑚+𝜈01^𝑑₊₁<(𝑚−1)𝑠²₁𝑚𝜒²_𝑚−1∶=𝐴₂. (33) Therefore, we have 𝐵₁+𝐵<₂^𝑑𝐴₁+𝐴₂. (34) Consequently, we have 𝐵₁+𝐵<₂^𝑑𝐴₁+𝐴₂. (35)
(II) We now show that the final conclusion holds. In fact, if we let 𝐶(𝑥,𝑦)=𝑐−(𝑥−𝑦), then 𝑝(𝑥)=𝑃𝐴₁𝑉−𝐴₂𝑃𝑈≤𝐶𝑥,𝑦=𝐸𝐴₁𝑉−𝐴₂𝑈≤𝐶𝑥,𝑦∣𝐴₁,𝐴₂Φ𝐶=𝐸𝑥,𝑦𝐴₁+𝐴₂, (36) where Φ(⋅) stands for the cumulative distribution function of a standard normal distribution and the last equation is due to the fact that 𝑈 and 𝑉 are independent normal distributions.
Similarly, for (24), we have lim_{𝜅01,𝜅02,𝜎01,𝜎02→0}𝑃𝐻₀𝑃∣𝑥,𝑦=𝐸𝐵₁𝑉−𝐵₂𝑈≤𝐶𝑥,𝑦∣𝐵₁,𝐵₂Φ𝐶=𝐸𝑥,𝑦𝐵₁+𝐵₂. (37)
Note that for each 𝐶(𝑥,𝑦) in (−∞,0),Φ(𝐶(𝑥,𝑦)/𝑅) is increasing in 𝑅∈(0,∞). Therefore, by (35), we have Φ𝐶𝑥,𝑦𝐵₁+𝐵₂^𝑑𝐶<Φ𝑥,𝑦𝐴₁+𝐴₂,as𝐶𝑥,𝑦<0. (38)
Consequently, we have 𝐸Φ𝐶𝑥,𝑦𝐵₁+𝐵₂Φ𝐶<𝐸𝑥,𝑦𝐴₁+𝐴₂,as𝐶𝑥,𝑦<0. (39)
In addition, by the symmetry of the 𝑡-distribution, it follows that 𝐶(𝑥,𝑦)<0 is equivalent to 𝑝(𝑥)<1/2. Therefore, by (36), (37), and (39), the conclusion of Theorem 3 holds.

The following theorem shows that, even for fixed 𝑚 and 𝑛 with 2<𝑚, 𝑛<∞, we still obtain the reconcilability of the frequentist and Bayesian evidence.

Theorem 4. As min{𝜈₀₁,𝜈₀₂}→∞, the conclusion of Theorem 3 holds for any fixed 𝑚 and 𝑛 with 2<𝑚, 𝑛<∞; that is, inf_{𝜋∈𝐺𝑐2}𝑃𝐻₀(𝑥)(𝑥)<1∣𝑥<𝑝,𝑎𝑠𝑝2. (40)

Proof. We still adopt the notations of Theorem 3. We first prove that, as 𝜈₀₂→∞, 𝐵<₁^𝑑𝐴₁. (41) By the proof of Theorem 3, we have 𝑃𝐵₁<𝛾=𝑃𝜒²_{𝑛+𝜈02+1}>(𝑛−1)𝑠²₂,𝑛𝛾+𝜖𝑛,𝛾 (42) where 𝜖=𝜈𝑛,𝛾⁰²+2𝑠²₂𝑛+𝜈₀₂.+2𝑛𝛾 (43) It is obvious that sup_𝜈02𝜖𝑛,𝛾=lim_𝜈02→∞𝜖=𝑠𝑛,𝛾²².𝑛𝛾 (44)
Let 𝑓_𝑛(𝑡) denote the density function of 𝜒²_𝑛. Then it is easy to see that 𝑓_𝑛(𝑡) reaches the maximum at 𝑡=𝑛−2 and that max_𝑡𝑓_𝑛(𝑡)=𝑓_𝑛(𝑛−2)→0, as 𝑛→∞. Therefore, as 𝜈₀₂ is sufficiently large, it holds that 𝑃𝐵₁<𝛾=𝑃𝜒²_{𝑛+𝜈02+1}>𝑠²₂𝛾+𝜖𝜈₀₂=𝑃𝐵₁<𝛾+𝜖𝜈₀₂ (45) for some 𝜖(𝜈₀₂)>0, where 𝜖(𝜈₀₂)→0, as 𝜈₀₂→∞, and 𝐵₁=𝑠²₂/𝜒²_{𝑛+𝜈02+1}.
Therefore, for any fixed 𝑛(2<𝑛<∞), as 𝜈₀₂→∞, we have 𝑃𝐵₁<𝛾=𝑃𝜒²_{𝑛+𝜈02+1}>𝑠²₂𝛾𝜒=𝑃^{2𝑛+𝜈02+1}−𝑛+𝜈₀₂+12𝑛+𝜈₀₂>𝑠+1₂₂/𝛾−𝑛+𝜈₀₂+12𝑛+𝜈₀₂𝑠+1=1−Φ²²/𝛾−𝑛+𝜈₀₂+12𝑛+𝜈₀₂+1−𝜖₁𝜈₀₂⟶1, (46) where 𝜖₁(𝜈₀₂)→0, as 𝜈₀₂→∞.
On the other hand, for any fixed 𝑛(2<𝑛<∞) and 𝛾≠0, we have 𝑃𝐴₁<𝛾=𝑃𝜒²_𝑛−1>(𝑛−1)𝑠²₂𝑛𝛾≤1−𝜖₀, (47) for some 𝜖₀>0. Therefore, as 𝜈₀₂→∞, by (46) and (47), we have 𝑃(𝐵₁<𝛾)−𝑃(𝐴₁<𝛾)>0. Furthermore, by (45), we have 𝑃𝐵₁<𝛾>𝑃𝐴₁<𝛾. (48) That is, 𝐵<₁^𝑑𝐴₁. (49) Similarly, for any fixed 𝑚(2<𝑚<∞), as 𝜈₀₁→∞, we have 𝐵<₂^𝑑𝐴₂. (50) Therefore, similar to Theorem 3, as min{𝜈₀₁,𝜈₀₂}→∞, for any fixed 𝑚 and 𝑛 with 2<𝑚, 𝑛<∞, we have 𝐵₁+𝐵<₂^𝑑𝐴₁+𝐴₂. (51) The rest part of the proof is similar to that of (II) of Theorem 3.

The following simulation results show that even for small and fixed values of 𝑚 and 𝑛 or 𝜈₀₁ and 𝜈₀₂, the generalized 𝑃 value and Bayesian evidence for testing the Behrens-Fisher problem are still reconcilable.

For fixed 𝜈₀₁=0.5 and 𝜈₀₂=1 and for 𝑠²₁=1 and 𝑠²₂=4, taking different values of 𝑚 and 𝑛, some results of comparing the 𝑃 value and lim_{𝜅01,𝜅02,𝜎01,𝜎02→0}𝑃(𝐻₀∣𝑥) are listed in Table 1.

Table 1 

𝑃 value and lim𝑃(𝐻₀∣𝑥) for testing the Behrens-Fisher problem for different 𝑚 and 𝑛.

For fixed 𝑚=8 and 𝑛=10 and for 𝑠²₁=1 and 𝑠²₂=4, taking different values of 𝜈₀₁ and 𝜈₀₂, we list some results of comparing 𝑃 value and lim_{𝜅01,𝜅02,𝜎01,𝜎02→0}𝑃(𝐻₀∣𝑥) in Table 2.

Table 2 

𝑃 value and lim𝑃(𝐻₀∣𝑥) for testing the Behrens-Fisher problem for different 𝜈₀₁ and 𝜈₀₂.

3. Conclusions

In the presence of the nuisance parameters, we study the reconcilability of the 𝑃 value and the Bayesian evidence in the one-sided hypothesis testing problem about normal means. For the problem of testing a normal mean where the nuisance parameter is present, it is shown that the Bayesian and frequentist lines of evidence are reconcilable. For the Behrens-Fisher problem, it is illustrated that if the sample sizes 𝑚 and 𝑛 tend to infinity, then for fixed prior parameters 𝜈₀₁ and 𝜈₀₂, both lines of evidence are reconcilable. Furthermore, it is illustrated that if the prior parameters 𝜈₀₁ and 𝜈₀₂ tend to infinity, then for any fixed sample sizes 𝑚 and 𝑛, lines of evidence are reconcilable. Simulation results show that even for small and fixed values of sample sizes 𝑚 and 𝑛 or for small values of prior parameters 𝜈₀₁ and 𝜈₀₂, the reconcilable conclusion of the Bayesian and frequentist evidence still holds.

This provides another illustration of testing situation where the Bayesian and frequentist evidence can be reconciled and may therefore to some extent prevent people from debasing or even dismissing 𝑃 values as evidence in hypothesis testing problems. Furthermore, our results of the reconcilability in the one-sided testing situations may help us to come to the idea that maybe it is arbitrary to assert the irreconcilability of the evidence in the two-sided (point or interval) hypothesis testing problems and perhaps we should be concerned more about the appropriateness of the methods we employ to tackle a two-sided hypothesis in both the frequentist and the Bayesian frameworks.

Acknowledgments

The work was supported by the Foundation for Training Talents of Beijing (Grant no. 19000532377), the Project of Construction of Innovative Teams and Teacher Career Development for Universities and Colleges Under Beijing Municipality (Grant no. IDHT20130505) and the Research Foundation for Youth Scholars of Beijing Technology and Business University (Grant no. QNJJ2012-03).