Research Article | Open Access
Sévérien Nkurunziza, Fuqi Chen, "Equivariance and Generalized Inference in Two-Sample Location-Scale Families", Journal of Probability and Statistics, vol. 2011, Article ID 474826, 16 pages, 2011. https://doi.org/10.1155/2011/474826
Equivariance and Generalized Inference in Two-Sample Location-Scale Families
We are interested in-typical Behrens-Fisher problem in general location-scale families. We present a method of constructing generalized pivotal quantity (GPQ) and generalized value (GPV) for the difference between two location parameters. The suggested method is based on the minimum risk equivariant estimators (MREs), and thus, it is an extension of the methods based on maximum likelihood estimators and conditional inference, which have been, so far, applied to some specific distributions. The efficiency of the procedure is illustrated by Monte Carlo simulation studies. Finally, we apply the proposed method to two real datasets.
In statistical problems involving nuisance parameters, the small-sample optimal solution may not be available. For example, for the difference between means of two exponential distributions, or two normal distributions with different variances, small sample-optimal test and confidence intervals do not exist (see, ). To overcome this problem, Tsui and Weerahandi  introduced the concept of generalized value (GPV) and generalized test variable (GTV). Further, Weerahandi  developed the concept of generalized pivotal quantity (GPQ) and generalized confidence interval (GCI). The GCI and GPV have been revealed to perform well for some small-sample problems where classical procedures are not optimal. For example, Weerahandi  applied the GCIs to the difference in two exponential means and two normal means. In addition, Bebu and Mathew  developed a generalized pivotal quantity for comparing the means and variances of a bivariate log-normal distribution.
In this paper, we present a method of constructing the GPQ and GTV in two-sample location-scale families. Also, we extend the method in Sprott  where the author applied conditional inference to some particular bivariate location-scale families. In the quoted book, the author uses the maximum likelihood estimator (MLE). However, it is well known that the MLE does not exist in some location and scale families. For more details, we refer to Pitman , and Gupta and Székely  among others.
Our proposed method is based on Pitman estimator that is the minimum risk equivariant estimator (MRE). It is noticed that, when MLE of a location parameter (or scale parameter) exists, it is an equivariant estimator. Indeed, the suggested method is more general, and our simulation studies show that it provides a high coverage probability, high power and preserves the nominal level of the test.
The rest of this paper is organized as follows. In Section 1, we present some background about generalized inference in location and scale family. We also establish in Section 1 the proposed generalized pivotal quantity and generalized test variable in two-sample location-scale family. Section 2 gives the main result of this paper. Namely, in this section, we present the algorithm of the proposed method. In Section 3, we discuss the application of the method in some specific location-scale families. Section 4 presents some simulation studies as well as analysis results of two real datasets. Finally, Section 5 gives discussion and concluding remarks. Details and technical results are outlined in two appendices.
2. Background and Preliminary Results
In this section, we present some concepts of generalized inference for the convenience of the reader. Also, we set up notation which is used in this paper. For more details about the concepts of GPQ, GTV, and GPV, the reader is referred to Tsui and Weerahandi , Weerahandi , and Krishnamoorthy et al.  among others. Let be i.i.d. random variables from the population probability density function (pdf) . Also, let be iid random variables from the population pdf . We assume that the two random samples and are independent. Also, let be a -column vector of unknown parameters (with ). Further, let be a -column vector function of with , and to simplify the notation, let where is the parameter of interest and is a vector of nuisance parameters. Let denote the sample space of possible values of , where , , and let denote the parameter space of . In addition, we denote as an observation from . Given this statistical model, two statistical problems about are considered.
First, we are interested in deriving confidence interval estimation of . Second, for a given , we consider the testing problem
Definition 2.1. Let be a function of , , , , , where . Then, the function is said to be a generalized pivotal quantity for if (1)given , the distribution of is free from unknown parameters; (2)the observed value, defined as , does not depend on the nuisance parameter .
Definition 2.2. The generalized test variable for , is defined as a function of , say , which satisfies the following requirements. (1) is free of . (2)For fixed , , , the distribution of is free of the nuisance parameter .(3)For fixed , and , is stochastically monotone in .
To make the connection between GPQ and GTV, it is noticed that the GTV can be derived from GPQ . In fact, if is a GPQ for then, is a GTV. For instance, if , we have . For more details, see Krishnamoorthy et al. . Also, the generalized value for the testing problem (2.1) is defined as . More specifically, for the case where , the GPV for the testing problem (2.1) becomes Thus, since the distribution of is free of any unknown parameters, the GPV for can be obtained from (2.3) by either analytical method or Monte Carlo simulation. We consider the case where , . Thus, we present the GPQ and GTV for the difference between two location parameters . On one hand, we are interested in deriving GCIs for . On the other hand, we consider solving the following testing problem: Let denote the GPQ for . For the testing problem (2.4), the generalized value is
2.1. Equivariance and Minimum Risk Equivariant Estimators
In this subsection, we give a brief background about the concept of equivariance and minimum risk equivariant estimators in location-scale family. For more details about this concept, we refer to Lehmann and Casella [8, page 171–173], Schervish [9, chapter 6] among others. To set up some notation, let be a random sample whose joint pdf can be written as where is a pdf which does not depend on and . Then, is said to be from the location-scale family with location parameter and scale parameter .
An estimator for the scale parameter is said to be equivariant if it satisfies An estimator for the location parameter is said to be equivariant if it satisfies Also, let be equivariant estimator for the scale (or location) parameter and let be its risk function, that is, the expected value of a certain loss function which is invariant under the scale (or location) transformation. Then, the estimator is said to be minimum risk equivariant estimator (MRE) if for any other equivariant estimator for , , we have
In this paper, the loss function under consideration is the quadratic error loss function, and in this case, the minimum risk equivariant estimator is also known as Pitman estimator (see Lehmann and Casella [8, pages 154–174]).
In particular, let and , denote the minimum risk equivariant estimator for and , , respectively. In this notation, the subscript refers to Pitman estimator. Further, let , denote the observed values of and , , respectively. We close this section by recalling the result which is used in computing , and , .
Theorem 2.3. Let be iid random sample from scale-location family with pdf , where and are unknown. Also, under quadratic error loss function, suppose that there exists an equivariant estimator with finite risk. Then, under quadratic loss function the MRE of and are, respectively
2.2. GPQ and GTV in Two-Sample Location-Scale Families
Let and be two-sample iid from the population pdfs respectively with ; ; where , , , unknown parameters, and and are pdfs. Then, the joint pdf of , is given by
We are interested in inference problems concerning the difference between the location parameters , with unknown. To set up notation, let , where By using equivariance property of and , , we derive the GPQ, GTV of . Indeed, let where , are defined by (2.12). Then, are GPQ for , . Using , , we derive the GPQ and GTV for as given by the following proposition.
Proposition 2.4. If the two samples are from the pdf in (2.10), the GPQ for is Furthermore, the GTV is .
Proof. Obviously, the observed value of is . Further, since and , are equivariant for and , respectively, by using Lemma A.3, we conclude that the distributions of , are not dependent on parameter. Therefore, the distribution of does not depend on parameter, and this completes the proof.
In the following section, we present an algorithm which is used in computing the GCI and GPV. The proposed algorithm extensively uses Proposition 2.5 and Corollary 2.6 given below. To the best of our knowledge, these two results are not in the existing literature. To set up notation, let , and let , where Further, let , let and let , .
Proposition 2.5. Assume that the two random samples are from the pdfs in (2.10). Then, conditionally to , , the joint pdf of is , where
Proposition 2.5 extends Corollary A.2 that is established in Appendix A. The proof follows from similar arguments as for Corollary A.2. Further, from Proposition 2.5, we establish Corollary 2.6 that gives the joint pdf of conditionally to , .
Proof. The proof follows directly from Proposition 2.5.
In general, the distributions of the GPQ in (2.14) do not have a closed form. Accordingly, Monte Carlo simulations are needed in order to compute numerically the distributions of . In this section, we present an algorithm which is used in computing the GCI and GPV for . The proposed algorithm is applicable to all members of location-scale families, and in particular, it is applicable to the normal family that is the most commonly discussed in the literature. To the best of our knowledge, there does not exist a similar algorithm in the literature.
The proposed GCI and GPV are obtained by using the following algorithm. (1)For a given dataset , using Theorem 2.3, compute , , the observed values of , , , respectively. (2)By using (2.15), compute , , and , . (3)Generate for . (4)From the pdf of , given in (2.18), determine and such that .(5)By using (2.14), compute . (6)Repeat from step (3) to (5), times (with large), and set the value of obtained at the th replicate, . (7)Find and as, respectively, and percentiles of . (8)Let denote the indicator function of the event . Using (2.3), estimate the GPV for by .
Remark 3.1. The equations in step (4) of the above algorithm do not generally give a closed-form solution. Thus, some numerical methods are needed in order to find the quantiles and . In this paper, we applied Newton's method.
Remark 3.2. For the normal sample case, the proposed algorithm produces the same solution as in Weerahandi . Indeed, at normal case, as established in Section 3, and have Student's distributions with, respectively, and degrees of freedom.
4. Some Cases of Two-Sample Location-Scale Families
In this section, we discuss the application of the proposed method to some specific two-sample location-scale families. More precisely, we discuss the application of the proposed method to the two-sample location-scale families for which MLEs do not exist. Also, in order to illustrate the fact that the proposed approach generalizes the method designed at normal case, we discuss briefly the two-sample normal families case.
4.1. Two-Sample Normal Case
Let and let . From (2.3), we have Under the model in (4.1), we illustrate the computation of GCI and GPV, based on the proposed GPQ. To set up notation, let If and , using Theorem 2.3, we have and . Also, ,, .
Then, using Proposition 2.4 and some computations, we have where stands for a Student't variate with degrees of freedom. Similarly and taking , we get,
4.2. Two Location-Scale Families Case Where MLE Does Not Exist
The second illustrative example is based on the result in Pitman . Namely, we consider families , , where
Pitman  proved that MLEs for , , do not exist. For the families in (4.7) and (4.8), the pdf of and , do not have a closed form and thus, the distribution of is obtained numerically by using the algorithm given in Section 2.
5. Simulation Study and Data Analysis
5.1. Simulation Study
In this section, we carry out intensive simulation studies in order to evaluate the performances of the suggested approach in small and moderate sample sizes. To this end, we generate 10000 two-samples from logistic distribution, from the distribution in (4.7), and from the distribution in (4.8). In order to save space, we report below the empirical coverage probability and the empirical power for the location-scale family given in (4.8). Namely, the simulated coverage probabilities of the 95% GCI are presented in Table 1, and the empirical powers of the proposed test are given in Table 2, at significance level .
In particular, concerning the GCI of , Table 1 shows that, for , the coverage probabilities are also relatively close to the nominal confidence level of 95%. Interestingly, the case of equal scale parameters and that of unequal scale parameters seem to provide similar results. Further, it is noticed that as the sample size increases, the coverage probability gets closer to the nominal confidence level (95%). Concerning the performance of the solution to the testing problem (2.4), Table 2 shows that the power function varies with different values of , , , , , and . In fact, from Figure 1, it can be seen that when , the powers are all approximately equal to 0.05. But on the left-hand side of 0, the power continually increases to 1 when the distance between and 0 increases. Also, in the right hand side, the power decreases to 0 as the distance increases. Furthermore, in the left hand side of 0, for each exact value of , the power increases as the sample size increases.
(a) If (the family in (4.8))
(b) If (the family in (4.8))
5.2. Illustrative Examples and Data Analysis
5.2.1. Normal Body Temperature Dataset
This dataset is found in Mackowiak et al. . In this dataset, a total number of 130 patients have been assigned, with 65 males and 65 females. Their body temperatures have been tested and recorded. Furthermore, it is already confirmed that the temperatures in these 2 gender groups are normally distributed. In particular, for the male group, one can consider , and for the female group, one can consider . From Table 3, a 95% GCI for is and thus, since the interval does not contain 0, there is a significant difference between the two location parameters. By applying (2.5) to the testing problem versus , the GPV is found to be 0.0133, and this result indicates that the null hypothesis should be rejected at 2% significant level, that is, this confirms that .
5.2.2. Cloud Seeding Dataset
The cloud seeding dataset consists of the amount of rainfall (in acre-feet) which have been recorded. The dataset is given in Krishnamoorthy, and Mathew . For this dataset, 26 clouds were randomly seeded with silver nitrate, and 26 others were unseeded. In the above quoted paper, the authors showed that lognormal model fits the dataset very well. Thus, we assume unseeded cloud group and seeded cloud group . We set and .
From Table 4, the GCI for indicates that the difference between the two location parameters is statistically significant. Also, for the testing problem versus , the GPV is 0.007 which indicates that . Note that this finding corroborates the result given in Krishnamoorthy and Mathew , where the authors concluded that is statistically different from .
In this paper, we proposed a solution of typical Behrens-Fisher problem in the general setting where two independent samples are from location-scale families. We presented a general statistical method for constructing GPQ and GTV for the difference between two location parameters of location-scale families. The proposed method is based on the minimum risk equivariant estimators which are known to be more general and more efficient than the MLEs. The simulation studies show that the proposed methods provide CIs and tests with high coverage probability and power, and the resulting tests preserve the significance level.
The proposed method applies to all members of the location-scale families, as opposed to the methods given in the literature, as Welch's method, which are designed only for the normal case. In addition to this generality, our method is at least as good as Welch's method in the normal Behrens-Fisher problem (see simulation results in Appendix B).
A. Technical Results and Proof of Proposition 2.5
In this subsection, we present some results which are useful in deriving Proposition 2.5. Recall that this last proposition is used in deriving Corollary 2.6 that plays a central role in the proposed algorithm as given in Section 2. For the sake of simplicity, the results are outlined for the case where , that is, when the two samples and are independent from location families with location parameters , .
Let be the equivariant estimator of , . Also, let where the two samples and are independent from location families with location parameters , .
Proposition A.1. Assume two random samples are from two independent location families and assume that relation (A.1) holds. Then , are ancillary statistics. Furthermore, the joint pdf of , is
Proof. From the fact that and are equivariant estimators for and , respectively, we conclude that , are ancillary statistics. Further, without loss of generality, assume that . Also, let us define , and by , . Then, since and are equivariant, and can be expressed as a function of , and thus, one can set , . Then Let , let , let , and let . We have . Also, let , . The joint pdf of is where is the Jacobian matrix. We have Therefore, from (A.4) and letting , , we get as stated in the proposition, and that completes the proof.
Proof. From (A.4), we directly get the conditional joint pdf of given , . By algebraic computations, we verify that the conditional pdf of corresponds to that stated in the corollary.
Lemma A.3. Let be a random sample from the location-scale family with location parameter and scale parameter . Also, let , be equivariant estimators for and , respectively. Then, the distributions of and do not depend on the parameters and .
Proof. Let , let , let , and let . Since and are equivariant for and , respectively, we have (see Lehmann and Casella [8, pages 171–173]) Hence, taking , , we obtain Further, since is from a location-scale family with location and scale parameters and respectively, the distribution of does not depend on parameter. Indeed, the joint pdf of can be written as where is a pdf which does not depend on and . Then, the joint pdf of is . Hence, the distributions of and do not depend on parameter. Therefore, from (A.8) and (A.9), we conclude that the distributions of and do not depend on parameter, and this completes the proof.
B. Simulation Results in Normal Samples Case
In this section we present some numerical results for the normal samples case. Indeed, the proposed approach generalizes the existing methods used in solving the well-known Behrens-Fisher problem.
For comparison purposes, we compared the proposed method with bootstrap. In particular, Table 6 shows that the proposed method dominates the bootstrap in small sample cases, and it is at least as good as the bootstrap in large sample cases. Also, we present in Table 5 the coverage probability obtained by using the Welch's approximation method for the normal case. Also, we present in Tables 7 and 8 the empirical powers obtained by using the Welch approximation method for the normal case. In summary, for the Behrens-Fisher problem with unbalanced sample sizes, the proposed confidence interval is at least as accurate as that given by Welch method. Further, the proposed test is at least as powerful as the Welch approximation test. In addition, the proposed method has the advantage of being useful for the more general statistical model of two samples from location-scale family.