Abstract

We conduct a systematic comparison of the performance of four commonly used value combination methods applied to panel unit root tests: the original Fisher test, the modified inverse normal method, Simes test, and the modified truncated product method (TPM). Our simulation results show that under cross-section dependence the original Fisher test is severely oversized, but the other three tests exhibit good size properties. Simes test is powerful when the total evidence against the joint null hypothesis is concentrated in one or very few of the tests being combined, but the modified inverse normal method and the modified TPM have good performance when evidence against the joint null is spread among more than a small fraction of the panel units. These differences are further illustrated through one empirical example on testing purchasing power parity using a panel of OECD quarterly real exchange rates.

1. Introduction

Combining significance tests, or values, has been a source of considerable research in statistics since Tippett [1] and Fisher [2]. (For a systematic comparison of methods for combining values from independent tests, see the studies by Hedges and Olkin [3] and Loughin [4].) Despite the burgeoning statistical literature on combining values, these techniques have not been used much in panel unit root tests until recently. Maddala and Wu [5] and Choi [6] are among the first who attempted to test unit root in panels by combining independent values. More recent contributions include those by Demetrescu et al. [7], Hanck [8], and Sheng and Yang [9]. Combining values has several advantages over combination of test statistics in that (i) it allows different specifications, such as different deterministic terms and lag orders, for each panel unit, (ii) it does not require a panel to be balanced, and (iii) observed values derived from continuous test statistics have a uniform distribution under the null hypothesis regardless of the test statistic or distribution from which they arise, and thus it can be carried out for any unit root test derived.

While the formulation of the joint null hypothesis (: all of the time series in the panel are nonstationary) is relatively uncontroversial, the specification of the alternative hypothesis critically depends on what assumption one makes about the nature of the heterogeneity of the panel. (Recent contributions include O’Connell [10], Phillips and Sul [11], Bai and Ng [12], Chang [13], Moon and Perron [14] and Pesaran [15].) The problem of selecting a test is complicated by the fact that there are many different ways in which can be false. In general, we cannot expect one test to be sensitive to all possible alternatives, so that no single value combination method is uniformly the best. The goal of this paper is to make a detailed comparison, via both simulations and empirical examples, of some commonly used value combination methods, and to provide specific recommendation regarding their use in panel unit root tests.

The plan of the paper is as follows. Section 2 briefly reviews the methods of combining values. Small sample performance of these methods is investigated in Section 3 using Monte Carlo simulations. Section 4 provides the empirical applications, and Section 5 concludes the paper.

2. Value Combination Methods

Consider the model Heterogeneity in both the intercept and the slope is allowed in (2.1). This specification is commonly used in the literature, see the work of Breitung and Pesaran [16] for a recent review. Equation (2.1) can be rewritten as where and .

The null hypothesis is and the alternative hypothesis is

Let be a test statistic for the th unit of the panel in (2.2), and let the corresponding value be defined as , where denotes the cumulative distribution function (c.d.f.) of . We assume that, under , has a continuous distribution function. This assumption is a regularity condition that ensures a uniform distribution of the values, regardless of the test statistic or distribution from which they arise. Thus, value combinations are nonparametric in the sense that they do not depend on the parametric form of the data. The nonparametric nature of combined values gives them great flexibility in applications.

In the rest of this section, we briefly review the value combination methods in the context of panel unit root tests. The first test, proposed by Fisher [2], is defined as which has an distribution with 2 degrees of freedom under the assumption of cross-section independence of the values. Maddala and Wu [5] introduced this method to the panel unit root tests, and Choi [6] modified it to the case of infinite .

Inverse normal method, attributed to Stouffer et al. [17], is another often used method defined as where is the c.d.f. of the standard normal distribution. Under , ~. Choi [6] first applied this method to the panel unit root tests assuming cross-section independence among the panel units. To account for cross-section dependence, Hartung [18] developed a modified inverse normal method by assuming a constant correlation across the probits , where . He proposed to estimate in finite samples by where and . The modified inverse normal test statistic is formed as where is a parameter designed to improve the small sample performance of the test statistic. Under the null hypothesis, . Demetrescu et al. [7] showed that this method was robust to certain deviations from the assumption of constant correlation between probits in the panel unit root tests.

A third method, proposed by Simes [19] as an improved Bonferroni procedure, is based on the ordered values, denoted by . The joint hypothesis is rejected if for at least one . This procedure has a type I error equal to when the test statistics are independent. Hanck [8] showed that Simes test was robust to general patterns of cross-sectional dependence in the panel.

The fourth method is Zaykin et al.’s [20] truncated product method (TPM), which takes the product of all those values that do not exceed some prespecified value . The TPM is defined as where is the indicator function. Note that setting leads to Fisher's original combination method, which could lose power in cases when there are some very large values. This can happen when some series in the panel are clearly nonstationary such that the resulting -values are close to 1, and some are clearly stationary such that the resulting values are close to 0. Ordinary combination methods could be dominated by the large values. The TPM removes these large values through truncation, thus eliminating the effect that they could have on the resulting test statistic.

When all the values are independent, there exists a closed form of the distribution for under . When the values are dependent, Monte Carlo simulation is needed to obtain the empirical distribution of . Sheng and Yang [9] modify the TPM to allow for a certain degree of correlation among the values. Their procedure is as follows.

Step 1. Calculate using (2.11). Set .

Step 2. Estimate the correlation matrix, , for values. Following Hartung [18] and Demetrescu et al. [7], they assume a constant correlation between the probits and , where and . can be estimated in finite samples according to (2.8).

Step 3. The distribution of is calculated based on the following Monte Carlo simulations.(a)Draw pseudorandom probits from the normal distribution with mean zero and the estimated correlation matrix, , and transform them back through the standard normal c.d.f., resulting in -values, denoted by . (b)Calculate . (c)If , increment A by one. (d)Repeat steps (a)–(c) B times.(e)The value for is given by .

3. Monte Carlo Study

In this section we compare the finite sample performance of the value combination methods introduced in Section 2. We consider β€œstrong” cross-section dependence, driven by a common factor, and β€œweak” cross-section dependence due to spatial correlation.

3.1. The Design of Monte Carlo

First we consider dynamic panels with fixed effects but no linear trends or residual serial correlation. The data-generating process (DGP) in this case is given by where for , . The initial values are set to be 0 for all . The individual fixed effect , the common factor , the factor loading , and the error term are independent of each other with i.i.d , i.i.d , i.i.d , and i.i.d .

Remark 3.1. Setting , we explore the properties of the tests under cross-section independence, and, with , we explore the performance of the tests under β€œhigh” cross-section dependence. In the latter case, the average pairwise correlation coefficient of and is 70%, representing a strong cross-section correlation in practice.

Next we allow for deterministic trends in the DGP and the Dickey-Fuller (DF) regressions. For this case is generated as follows: with i.i.d and i.i.d . This ensures that has the same average trend properties under the null and the alternative hypotheses. The errors are generated according to (3.2) with , representing the scenario of high cross-section correlation.

To examine the impact of residual serial correlation, we consider a number of experiments, where the errors in (3.2) are generated as with i.i.d . Following Pesaran [15], we choose i.i.d for positive serial correlations and i.i.d for negative serial correlations. We use this DGP to check the robustness of the tests to alternative residual correlation models and to the heterogeneity of the coefficients, .

Finally we explore the performance of the tests under spatial dependence. We consider two commonly used spatial error processes: the spatial autoregressive (SAR) and the spatial moving average (SMA). Let be the error vector in (3.1). In SAR, it can be expressed as where is the spatial autoregressive parameter, is an known spatial weights matrix, and is the error component which is assumed to be distributed independently across cross-section dimension with constant variance . Then the full covariance matrix is where . In SMA, the error vector can be expressed as with being the spatial moving average parameter. Then the full covariance matrix becomes Without loss of generality, we let . We consider the spatial dependence with and . The average pairwise correlation coefficient of and is 4%–22% for SAR and 2%–8% for SMA, representing a wide range of cross-section correlations in practice. The spatial weight matrix is specified as a β€œ1 ahead and 1 behind” matrix with the th row, , of this matrix having nonzero elements in positions and . Each row of this matrix is normalized such that all its nonzero elements are equal to 1/2.

For all of DGPs considered here, we use where indicates the fraction of stationary series in the panel, varying in the interval 0-1. As a result, changes in allow us to study the impact of the proportion of stationary series on the power of tests. When , we explore the size of tests. We set , 0.5 and 0.9 to examine the power of the tests under heterogeneous alternatives. The tests are one-sided with the nominal size set at 5% and conducted for all combinations of and , 50, and 100. (We also conduct the simulations with the nominal size set at 1% and 10%. The results are qualitatively similar to those at the 5% level, and thus are not reported here.) The results are obtained with MATLAB using simulations. To calculate the empirical critical value for the modified TPM, we run additional replications within each simulation.

We calculate the augmented Dickey-Fuller (ADF) statistics. The number of lags in the ADF regressions is selected according to the recursive -test procedure. (Start with an upper bound, , on . If the last included lag is significant, choose , if not, reduce by one until the last lag becomes significant. If no lag is significant, set . The 10 percent level of the asymptotic normal distribution is used to determine the significance of the last lag.) As shown in the work of Ng and Perron [21], this sequential testing procedure has better size properties than those based on information criteria in panel unit root tests. The values in this paper are calculated using the response surfaces estimated in the study by Mackinnon [22].

3.2. Monte Carlo Results

We compare the finite sample size and power of the following tests: Maddala and Wu’s [5] original Fisher test (denoted by ), Demetrescu et al.’s [7] modified inverse normal method (denoted by ), Hanck's [8] Simes test (denoted by ), and Sheng and Yang [9]'s modified TPM (denoted by ). The results in Table 1 are obtained for the case of cross-section independence for a benchmark comparison. Tables 2 and 3 consider the cases of cross-section dependence driven by a single common factor with the trend and residual serial correlation. Table 4 reports the results with spatial dependence. Given the size distortions of some methods, we also include the size-adjusted power in Tables 5, 6, and 7. Major findings of our experiments can be summarized as follows.(1)In the absence of clear guidance regarding the choice of , we try 10 different values, ranging from 0.05, 0.1, 0.2, …, up to 0.9. Our simulation results show that tends to be slightly oversized with a small but moderately undersized with a large and that its power does not show any clear patterns. We also note that yields similar results as varies between 0.05 and 0.2. In our paper we select . (To save space, the complete simulation results are not reported here, but are available upon request.) (2)With no cross-section dependence, all the tests yield good empirical size, close to the 5% nominal level (Table 1). As expected, test shows severe size distortions under cross-section dependence driven by a common factor or by spatial correlations. For a common factor with no residual serial correlation, while test is mildly oversized and test is slightly undersized, test shows satisfactory size properties (Table 2). The presence of serial correlation leads to size distortions for all statistics when is small, which even persist when for and tests. On the contrary, and tests exhibit good size properties with and 100 (Table 3). Under spatial dependence, test performs the best in terms of size, while and tests are conservative for large (Table 4). (3)All the tests become more powerful as increases, which justifies the use of panel data in unit root tests. When a linear time trend is included, the power of all the tests decreases substantially. Also notable is the fact that the power of tests increases when the proportion of stationary series increases in the panel. (4)Compared to the other three tests, the size-unadjusted power of test is somewhat disappointing here. An exception is that, when only very few series are stationary, test becomes most powerful. When the proportion of stationary series in the panel increases, however, test is outperformed by other tests. For example, in the case of no cross-section dependence in Table 1 with , , and , the power of test is 0.156, and, in contrast, all other tests have power close to 1. (5)Because test has severe size distortions, we only compare and tests in terms of size-adjusted power. (The power is calculated at the exact 5% level. The 5% critical values for these tests are obtained from their finite sample distributions generated by 2000 simulations for sample size , 50, and 100. Since Hanck’s [8] test does not have an explicit form of finite sample distribution, we do not calculate its size-adjusted power.) With the cross-section dependence driven by a common factor, test tends to deliver higher power for but lower power for than test (Tables 5 and 6). Under spatial dependence, however, the former is clearly dominated by the latter in most of the time. This is especially true for SAR process, where test exhibits substantially higher size-adjusted power than test (Table 7).

4. Empirical Application

Purchasing Power Parity (PPP) is a key assumption in many theoretical models of international economics. Empirical evidence of PPP for the floating regime period (1973–1998) is, however, mixed. While several authors, such as Wu and Wu [23] and Lopez [24], found supporting evidence, others [10, 15, 25] questioned the validity of PPP for this period. In this section, we use the methods discussed in previous sections to investigate if the real exchange rates are stationary among a group of OECD countries.

The log real exchange rate between country and the US is given by where is the nominal exchange rate of the th country's currency in terms of US dollar and and are consumer price indices in the US and country , respectively. All these variables are measured in natural logarithms. We use quarterly data from 1973 : 1 to 1998 : 2 for 27 OECD countries, as listed in Table 8. (Two countries, Czech Republic and Slovak Republic, are excluded from our analysis, since their data span a very limited period of time, starting at 1993 : 1.) All data are obtained from the IMF's International Financial Statistics. (Note that, for Iceland, the consumer price indices are missing during 1982:Q1–1982:Q4 in the original data. We filled out this gap by calculating the level of CPI from its percentage changes in the IMF database.)

As the first stage in our analysis we estimated individual ADF regressions: The null and alternative hypotheses for testing PPP are specified in (2.3) and (2.4), respectively. The selected lags and the values are reported in Table 8. The results in the left panel show that the ADF test does not reject the unit root null of real exchange rate at the 5% level except for New Zealand. As a robustness check, we investigated the impact of a change in numeraire on the results. The right panel reports the estimation results when the Deutsche mark is used as the numeraire. Out of 27 countries, only 5β€”Mexico, Iceland, Australia, Korea, and Canadaβ€”reject the null of unit root at the 5% level.

As is well known, the ADF test has low power with a short time span. Exploring the cross-section dimension is an alternative. However, if a positive cross-section dependence is ignored, panel unit root tests can also lead to spurious results, as pointed out by O’Connell [10]. As a preliminary check, we compute the pairwise cross-section correlation coefficients of the residuals from the above individual ADF regressions, . The simple average of these correlation coefficients is calculated, according to Pesaran [26], as The associated cross-section dependence () test statistic is calculated using In our sample is estimated as 0.396 and 0.513 when US dollar and Deutchemark are considered as the numeraire, respectively. The statistics, 71.137 for the former and 93.368 for the latter, strongly reject the null of no cross-section dependence at the conventional significance level.

Now turning to panel unit root tests, Simes test does not reject the unit root null, regardless of which numeraire, US dollar or Deutchemark, is used. However, the evidence is mixed, as illustrated by other test statistics. For 27 OECD countries as a whole, we find substantial evidence against the unit root null with Deutchemark but not with US dollar. In summary, our results from panel unit root tests are numeraire specific, consistent with Lopez [24], and provide mixed evidence in support of PPP for the floating regime period.

5. Conclusion

We conduct a systematic comparison of the performance of four commonly used -value combination methods applied to panel unit root tests: the original Fisher test, the modified inverse normal method, Simes test, and the modified TPM. Monte Carlo evidence shows that, in the presence of both β€œstrong” and β€œweak” cross-section dependence, the original Fisher test is severely oversized but the other three tests exhibit good size properties with moderate and large . In terms of power, Simes test is useful when the total evidence against the joint null hypothesis is concentrated in one or very few of the tests being combined, and the modified inverse normal method and the modified TPM perform well when evidence against the joint null is spread among more than a small fraction of the panel units. Furthermore, under spatial dependence, the modified TPM yields the highest size-adjusted power. We investigate the PPP hypothesis for a panel of OECD countries and find mixed evidence.

The results of this work provide practitioners with guidelines to follow for selecting an appropriate combination method in panel unit root tests. A worthwhile extension would be to develop bootstrap value combination methods that are robust to general forms of cross-section dependence in panel data. This issue is currently under investigation by the authors.

Acknowledgment

The authors have benefited greatly from discussions with Kajal Lahiri and Dmitri Zaykin. They also thank the guest editor Mike Tsionas and an anonymous referee for helpful comments. The usual disclaimer applies.