Temporal Characteristics of Heat Waves and Cold Spells and Their Links to Atmospheric Circulation in EURO-CORDEX RCMs
We study summer heat waves and winter cold spells and their links to atmospheric circulation in an ensemble of EURO-CORDEX RCMs in Central Europe. Results of 19 simulations were compared against observations over 1980–2005. Atmospheric circulation was represented by circulation types and supertypes derived from daily gridded mean sea level pressure. We examined observed and simulated characteristics of hot and cold days (defined using percentiles of temperature anomalies from the mean annual cycle) and heat waves and cold spells (periods of at least three hot/cold days in summer/winter). Although the ensemble of RCMs reproduces on average the frequency and the mean length of heat waves and cold spells relatively well, individual simulations suffer from biases. Most model runs have an enhanced tendency to group hot/cold days into sequences, with several simulations leading to extremely long heat waves or cold spells (the maximum length overestimated by up to 2-3 times). All simulations also produce an extreme winter season with (often considerably) higher number of cold days than in any observed winter. The RCMs reproduce in general the observed circulation significantly conducive to heat waves and cold spells. Zonal flow reduces the probability of temperature extremes in both seasons, while advection of warm/cold air from the south-easterly/north-easterly quadrant plays a dominant role in developing heat waves/cold spells. Because of these links, the simulation of temperature extremes in RCMs is strongly affected by biases in atmospheric circulation. For almost all simulations and all circulation supertypes, the persistence of supertypes is significantly overestimated (even if the frequency of a given supertype is underestimated), which may contribute to development of too-long heat waves/cold spells. We did not identify any substantial improvement in the EURO-CORDEX RCMs in comparison to previous ENSEMBLES RCMs, but the patterns of the biases are generally less conclusive as to general RCMs’ drawbacks.
Extreme temperature events, namely, heat waves (HWs) in summer and cold spells (CSPs) in winter, are atmospheric phenomena with high impacts on natural ecosystems and human society in midlatitudes. Excess human and animal morbidity and mortality, decreased agricultural production and demanding energy supply are among the main effects taking part in overall societal and economic losses (e.g., [1–6]).
Continued changes of climate observed over recent decades are manifested not only in an increase in global temperature but in changes in all components of the climate system . To understand such changes at regional scales and how they project to influence future climate, including characteristics of HWs and CSPs, regional climate models (RCMs) have been developed and used. However, the reliability of RCM simulations has been considerably limited by the issue that they often fail to reproduce various aspects of the historical climate. Besides other things, RCMs tend to inherit inadequacies of their driving global climate models (GCMs), including atmospheric circulation . It is therefore important to evaluate RCM simulations for the recent or historical climate in order to identify their biases, which may contribute to improvements and further development of the models as well as enhancing interpretation of future climate scenarios, including their uncertainties and limitations.
The validation is particularly important for high-impact events such as HWs and CSPs, and for links between multiple climate characteristics, including those between atmospheric circulation and surface climate variables. That is what the present study focuses on. Plavcová and Kyselý  reported overly persistent large-scale atmospheric circulation as a factor contributing to overestimated frequency and duration of HWs and CSPs in RCM simulations from the ENSEMBLES project (http://ensembles-eu.metoffice.com, ) over the central European region. The present analysis deals with an ensemble of RCMs from the Coordinated Regional-climate Downscaling Experiment over Europe (EURO-CORDEX, http://www.cordex.org/, ) and evaluates whether the newer generation of RCMs evinces improvements in reproducing characteristics of extreme temperature events and their links to atmospheric circulation.
2. Data and Methods
2.1. Model and Observed Data
The RCM simulations examined in this study were obtained from the database of the EURO-CORDEX project. We investigated 19 historical simulations of 4 RCMs over the control period of 1980–2005. The RCM simulations and their acronyms used throughout the paper are given in Table 1. This ensemble allowed us to analyse the influence of (i) the RCM formulation (4 RCMs included CCLM4, ALADIN53, RACMO22E, and RCA4) as well as (ii) the driving data (4 different GCMs and the “perfect/observed” ERA-INTERIM reanalysis), and (iii) the role of spatial resolution (two sets of RCA runs with 0.11 deg and 0.44 deg resolutions). Model results are compared against the high-resolution European gridded data set E-OBS (version 16; ) with both 0.22 deg and 0.44 deg resolutions for temperature (see Section 2.2 for the definition of temperature characteristics), and ERA-INTERIM reanalysis  for sea level pressure data (Section 2.3). The choice of the 1980–2005 period is based on availability of the data since it represents an intersection of the start of the RCM runs driven by ERA-INTERIM (in 1980) and the end of the historical runs (in 2005). We use daily maximum and minimum temperature, and daily mean sea level pressure.
2.2. Definition of Heat Waves and Cold Spells
Daily temperature series were first averaged over the central European region defined as the area 48.5°–51.5°N and 12°–18°E (Figure 1). Anomalies of daily maximum (TX) and minimum (TN) temperatures from their mean annual cycles (smoothed by 7-day running means) were then computed in all data sets. Hot (cold) days were defined as days with TX (TN) above (below) the 90% (10%) quantile of their empirical distributions in summer (winter). Summer was considered as June–August (JJA) and winter as December–February (DJF). The quantiles were calculated for the observed data and each individual RCM run.
Heat waves (HWs) are defined as periods of at least three hot days in JJA. While the first two days of a HW must be hot days (i.e., two consecutive hot days), all subsequent hot days are attached to a HW as long as there is no break longer than 1 day with TX below the threshold. This definition is a slight modification of that used in  where the first three hot days had to be consecutive. The current definition allows to better differentiate between the days before a HW and the first days of that HW since the cases when a HW is preceded by two hot days followed by a single day with a below-threshold temperature are now included in a given HW instead of being considered as days preceding that HW. There must be at least two days between the end of a HW and the beginning of the next HW, obviously.
Cold spells (CSPs) are defined analogously using cold days in winter. Days of a HW (CSP) are omitted if they overlap the end of the summer (winter) season.
2.3. Characteristics of Atmospheric Circulation
Atmospheric circulation is represented by 27 circulation types as in . The circulation types are defined using three circulation indices: flow direction, strength, and vorticity [14, 15], derived from daily gridded mean sea level pressure data at the 16 grid points shown in Figure 1 in terms of equations provided in . Since frequencies of some circulation types are low, we grouped the types into “supertypes” [16, 17] in order to obtain more robust results in analysing the persistence of atmospheric circulation. We defined 6 supertypes: northerly, southerly, easterly, westerly, cyclonic, and anticyclonic. Each supertype is comprised of 9 circulation types with a given flow characteristic (for example, the southerly supertype is comprised of S, SE, SW, CS, CSE, CSW, AS, ASE, and ASW types, while the cyclonic supertype is comprised of C, CN, CNE, CE, CSE, CS, CSW, CW, and CNW types) as shown in Table 2.
Types conducive to HWs/CSPs are evaluated in terms of an efficiency coefficient calculated as the ratio of the relative frequency of a given circulation type before or during HWs/CSPs to its long-term mean seasonal frequency . Statistical significance of the efficiency coefficients was tested using the block resampling method [9, 19]. Values of the coefficient significantly higher than 1 define types considered conducive to HWs/CSPs, while values significantly lower than 1 define types not favourable for the development of HWs/CSPs.
3.1. Characteristics of Observed and Simulated Heat Waves and Cold Spells
The numbers of HWs/CSPs over 1980–2005 as well as their mean length are reproduced relatively well if the mean of the ensemble of RCM simulations is considered (Table 3, Figure 2). However, individual simulations suffer from biases in temporal characteristics of these extreme temperature periods. Some model runs tend to overestimate grouping of hot/cold days into sequences since they produce more HWs/CSPs (larger-than-10% overestimation is denoted by + in Table 3) and higher percentage of hot/cold days within HWs/CSPs (larger-than-10% overestimation concerns 11 RCMs for HWs and 9 RCMs for CSPs), and almost all RCMs overestimate the maximum length of HWs/CSPs. In several RCM simulations, extremely long HWs or CSPs occur, with the maximum length overestimated by 2-3 times in comparison to observations (e.g., a 37-day-long HW in ALADIN53_I_11, compared to observed maximum length of 11 days; a 30-day-long CSP in RCA4_M_11, compared to observed maximum length of 14 days).
Since the definition of hot/cold days uses a relative threshold (90th/10th quantile), there is a constant number of hot/cold days in the observed and simulated data, so the aforementioned overestimated characteristics of HWs/CSPs are closely related to the enhanced grouping of hot/cold days in model simulations. The tendency to group cold days is further manifested by the fact that an extreme winter season with a higher number of cold days than in any observed winter is found in all model simulations (Figure 3). In some cases, around half of the winter days in that season are classified as cold (e.g., 52 cold days in RCA_M_11, and 45 cold days in ALADIN53_I_11).
In the next step, we evaluate the effects of large-scale atmospheric circulation on the clustering of hot and cold days and on the overestimated length of HWs and CSPs.
3.2. Circulation Types Conducive to Heat Waves and Cold Spells
The RCMs reproduce in general the observed circulation significantly conducive to HWs and CSPs (Figure 4). Onset of a HW (evaluated using two days before the beginning of a HW) is often linked to strong anticyclonic flow (conducive to HWs in almost all RCMs) or an anticyclonic flow with easterly and/or southerly component. The advection of warm air from the south-easterly quadrant plays dominant role during HWs (types SE and S are conducive to HWs in almost all RCMs). Circulation types conducive to CSPs are typically those with easterly and/or northerly flows. While both anticyclonic and cyclonic types occur before the onset of a CSP (the number of types conducive to CSPs is larger in both observed data and RCMs in comparison to HWs, so the overall picture is less clear), days of CSPs are linked primarily to anticyclonic flows. This pattern is captured reasonably well in the RCMs, including the higher diversity of circulation types supporting the development of CSPs. Zonal westerly (for CSPs also southwesterly, for HWs northwesterly) flow significantly reduces the probability of extreme temperature events in both seasons, and this is captured in the RCMs much better in winter than summer, probably due to generally stronger zonal circulation over Europe.
The diverse impacts of the easterly and westerly flow on development of extreme temperature events are indicated also in seasonal statistics. If observed total frequencies of circulation types (grouped into supertypes) and the total number of days in HWs/CSPs in given seasons are compared, there are significant links: more westerly flow days lead to fewer days in HWs/CSPs, while more easterly flow days lead to more days in HWs/CSPs (Table 4). These correlations are more significant in winter, and they are more pronounced in the RCMs than in the observed data, particularly for the easterly flow in winter for which all RCMs (except for RCA4_C_44) yield a higher correlation.
Focusing on persistence of atmospheric circulation, the mean length of sequences of circulation supertypes in summer is significantly overestimated for almost all RCM simulations and all supertypes (Table 5). In winter, the RCMs strongly and significantly overestimate the mean length of the westerly flow (by up to 80%), and they tend to overestimate also the mean length of the other supertypes (Table 5). Only exception is the mean length of the circulation supertypes shorter in an RCM than observed data; this concerns mainly southerly flow in summer and anticyclonic flow in winter.
The ensemble of 19 simulations (both reanalysis-driven and GCM-driven) of the newest generation of RCMs produced within the EURO-CORDEX project was evaluated for the central European region over the 1980–2005 period. We focused on models’ ability to simulate the frequency and length of summer heat waves and winter cold spells and their links to atmospheric circulation.
The ensemble mean (or median) of the 19 simulations was found capable of capturing observed temporal characteristics of HWs and CSPs. Although there is a relatively wide spread among the individual runs, on average, the ensemble reproduced the frequencies of HWs and CSPs as well as their mean length. An indication from the ensemble mean that the RCMs suffer from a common feature (and drawback) is a higher percentage of hot/cold days that occur within HWs/CSPs and the overestimation of the length of the longest HWs/CSPs. The enhanced grouping of days with extreme temperature anomalies (note that according to the definition, hot/cold days represent 10% of the summer/winter days in all data sets) is demonstrated particularly by the strong overestimation of the number of cold days in the “coldest” season detected in all RCM simulations. The ensemble-mean results show that by averaging the model biases, one can get the best reproduction of observed patterns . We also found an improvement in capturing the frequencies of HWs and CSPs in the ensemble mean of the CORDEX RCMs compared to ENSEMBLES simulations: the overestimation decreased by half (cf. ). On the other hand, the grouping of hot/cold days into HWs/CSPs (represented by the percentage of hot/cold days that occur within HWs/CSPs) remains overestimated (although there are some exceptions in individual models).
There is no “perfect” RCM with respect to the frequency of HWs and CSPs: for each RCM, at least one simulation substantially overestimates the occurrence of HWs or CSPs. The same holds true for the simulations with the same driving data. Even an RCM simulation driven by reanalysis (RCA4_I_44) strongly overestimates the number of HWs. While simulations driven by CNRM-CM5 are relatively good at reproducing the frequency of summer HWs, they are among the worst for winter CSPs.
With respect to the spatial resolution, although there are very small differences between the two E-OBS series with 0.44 deg and 0.22 deg resolutions, the RCA4 runs reveal substantial differences and even common patterns when comparing simulations with the same driving data differing in spatial resolution: (i) RCA4 runs with the 0.44 deg resolution simulate more HWs (or at least the same number of HWs) as the runs with the finer resolution, while the 0.11 deg runs tend to overestimate the length of the longest HW; (ii) RCA4 runs with the 0.44 deg resolution tend to produce more days in HWs than the 0.11 deg runs, and this link is opposite for winter CSPs. It remains open whether an improved simulation of the frequency of HWs (and days in HWs) in finer resolution is due to a better reproduction of summer convection, since Kotlarski et al.  identified that RCA overestimates precipitation at the 0.11 deg grid resolution compared to 0.44 deg over Europe.
An interesting finding is that some models tend to yield opposite biases for summer warm and winter cold extremes. ALADIN53 belongs among them since it produces significantly longer mean length of HWs but shorter mean length of CSPs in comparison to observations. Also, all simulations driven by MPI-ESM-LR lead to a higher-than-observed percentage of isolated hot days but a lower percentage of isolated cold days. This correspondence also reveals the influence of the driving data and the links to the atmospheric circulation, respectively.
Since air temperature anomalies are related to large-scale atmospheric circulation over Central Europe , the links between circulation and temperature could explain some models’ characteristics found. Especially winter-time temperature strongly depends on atmospheric flow [22, 23], and there is also a stronger influence of the driving data on circulation in nested RCMs in winter .
However, the CORDEX RCMs did not convincingly improve the reproduction of the frequency and persistence of circulation types and supertypes compared to the ENSEMBLES RCMs  and still suffer from biases (Figure 5). Above all, there is an overestimation of westerly flow in winter—a pattern well-known and described for climate models over Europe [24, 25]. This overestimation is strongest for runs driven by EC-EARTH (by 21–37%). In comparison to ERA-INTERIM, all RCMs have a tendency to produce fewer anticyclonic-while more cyclonic-flow days in winter. Even runs driven by reanalysis suffer from these biases. The influence of the driving GCM is indicated, for example, in the strong underestimation of anticyclonic-westerly flow in winter common to all runs driven by the MPI-ESM-LR GCM, while all runs driven by EC-EARTH overestimate the frequency of this circulation type. In summer, runs driven by CNRM-CM5 have a common tendency to overestimate cyclonic circulation while they underestimate anticyclonic flow days.
Biases in representing the persistence of atmospheric circulation, detected for the ENSEMBLES RCMs , remain present in the CORDEX RCMs, too. The common tendency to overestimate the duration of sequences of days with the same circulation supertype can only partially be explained by the higher frequency of some supertypes. The mean length of sequences of anticyclonic flow days in summer is overestimated (except for one case statistically significantly), even though most RCMs underestimate the overall occurrence of anticyclonic circulation. In winter, all RCMs underestimate appreciably the frequency of anticyclonic supertype, and still the RCMs’ median length of anticyclonic sequences is longer than the observed mean. As shown in previous studies, persistence of circulation patterns influences warm and cold temperature anomalies and therefore also conditions for development of HWs and CSPs [9, 26], and this is especially the case for blocking situations allowing meridional advection of warm or cold air masses . We detected links between the westerly/easterly flows and the number of HW/CSP days also from the seasonal statistics. These links are more pronounced in the RCMs than observed data and more significant in winter. Thus, a good representation of atmospheric circulation in climate models is important for the proper reproduction of periods of extreme temperatures.
Circulation types conducive to HWs and CSPs were identified by evaluating the relative frequency of HW-/CSP-days within a given type and within the whole season (i.e., the efficiency coefficient). All RCMs are able to capture the observed basic patterns and no type identified as conducive to HWs/CSPs in the observed data was significantly unfavourable (i.e., the efficiency coefficient significantly < 1) in any RCM, and vice versa. The specific differences among the RCMs in Figure 3 partly represent also the sensitivity of the significance testing of the efficiency coefficient which can be influenced by the biases in frequencies and overall small sample sizes of some circulation types in individual models. Prolonged temperature extremes (both HWs and CSPs) are primarily linked to easterly/southeasterly circulation.
According to the definition used, HWs and CSPs consisted of sequences of hot and cold days (allowing one-day exceptions), respectively. Hot/cold days represent 10% of all days with the largest Tmax/Tmin anomalies from the smoothed annual cycle in the observed data and individual models. This percentile-based definition is less affected by the model temperature bias than the absolute-threshold definition , and it is also more suitable for studying links between surface temperature and atmospheric circulation, as the effect of the mean annual cycle is suppressed. Lhotka and Kyselý  used the fixed threshold for defining cold days over Central Europe (−10°C, which roughly corresponds to the 10% quantile of regionally averaged wintertime Tmin in E-OBS). However, since RCMs suffer from a negative temperature bias, cold days were detected even under zonal flow in some models, which is in contradiction to observations. On the other hand, the percentile-based threshold set for each model brings other issues. There is a constant number of hot/cold days in all models (∼9 days per season on average), and the identification of HWs/CSPs depends on the grouping of these days with no matter of their severity (which can be evaluated separately).
Interestingly, we found that all RCM runs examined have a tendency to group cold days into seasons. While the maximum number of cold days in one season is 25 in the observed data, it ranges between 27 and 52 (with the mean of 34) in individual models. Namely, RCA_M_11 simulates a winter season in which 58% of days were detected as cold. This winter also led to the longest CSP identified, with the length of 30 days (note that some of the days in CSPs may have above-threshold temperature as long as there is no more than one such day in a row, see the definition of CSPs). Surprisingly, this model run (RCA_M_11), on the other hand, produces the smallest number of hot days in the warmest summer season (with the highest number of hot days) among all RCMs. This tendency towards “nongrouping” of hot days by this model, which sharply contrasts with the enhanced tendency to group cold days, is reflected in the lowest number of HWs and their mean duration. One may also identify an influence of the MPI-ESM-LR driving GCM on “nongrouping” of hot days, since all runs driven by this GCM produce fewer hot days that occur within HWs, and fewer HWs with shorter mean length than observed. In winter, there are relatively uniform percentages of cold days within CSPs among all runs driven by the same GCM for HadGEM2-ES and MPI-ESM-LR. This is an illustration of the links between climate characteristics and how biases in large-scale atmospheric circulation (which may propagate from the driving GCMs, ) influence biases in extreme temperature characteristics.
The reason for the enhanced tendency of the RCMs to produce an extremely cold winter season with longer and more severe CSPs than observed, which is a general feature within the examined ensemble, may be a too-strong (and overly persistent) interruption of the prevailing zonal flow, and this issue deserves further investigation.
As stated above, we found an improvement in reproducing the frequencies of HWs and CSPs if comparing the ensemble mean of the CORDEX and ENSEMBLES RCMs (cf. ). However, the spread of the biases in individual runs remains wide for both HWs and CSPs characteristics and also for circulation type frequencies. We note that in this analysis, an improved definition of HWs and CSPs was used (treating the cases when a HW/CSP is preceded by two hot/cold days followed by a single day that does not meet the criteria for hot/cold days as a single HW/CSP); however, this slight modification cannot qualitatively influence the comparison. On the other hand, the selection of ensemble members and especially its larger size could be a limitation for the direct comparison of the CORDEX and ENSEMBLES outputs.
We introduced an improved definition of heat waves (HWs) and cold spells (CSPs) for the study of temporal characteristics of these extreme temperature events and their links to atmospheric circulation in climate models and evaluated an ensemble of 19 RCM simulations (1980–2005) from the EURO-CORDEX project over Central Europe. The main conclusions of the study can be summarized as follows:(i)The frequencies of HWs and CSPs as well as their mean length are reproduced relatively well by the ensemble mean. The general overestimation of their frequencies reported for ENSEMBLES simulations  decreased, but the RCMs still have a tendency to produce too-long HWs and CSPs, and there is a wide spread among individual runs (and therefore large biases in some of the simulations).(ii)The RCMs have an enhanced tendency to group hot and cold days into sequences. All RCMs produce an extreme winter season with a higher number of cold days than in any observed winter, and some RCMs simulate extremely long CSPs. This tendency is less general but still evident for summer HWs.(iii)Atmospheric circulation significantly conducive to HWs and CSPs is generally reproduced well in the RCMs: anticyclonic flow (often with easterly and/or southerly component) typical for onset of a HW while advection of warm air from the south-easterly quadrant for the duration of a HW, and easterly and/or northerly flow for onset of a CSP while anticyclonic flow for the duration of a CSP. Zonal westerly flow significantly reduces the probability of extreme temperature events in both seasons.(iv)On the other hand, the CORDEX RCMs did not convincingly improve the reproduction of the frequency and persistence of the large-scale circulation compared to the ENSEMBLES RCMs. The persistence is significantly overestimated for almost all RCM simulations and all circulation supertypes, even in cases when frequencies of given supertypes are lower than observed, and this overestimation may contribute to enhanced grouping of hot/cold days and development of too-long HWs/CSPs.(v)There is no “perfect” RCM, no generally better driving data (GCM), and no clear benefit of finer spatial resolution as to reproduction of temporal characteristics of extreme temperature periods and their links to large-scale circulation.
As stated in , evaluation of the EURO-CORDEX ensembles generally confirms the RCM bias characteristics identified by previous studies based on the ENSEMBLES data and suggests that a significant improvement of climate models is a long-distance run. A finer resolution itself (without explicitly resolving convection, or other improvements in physical mechanisms) does not guarantee better performance or other benefits of the model outputs, as demonstrated in our study by examining the RCA4 runs with two different resolutions. An improved representation of atmospheric circulation and its persistence in climate models and more realistic land-atmosphere coupling (related to precipitation patterns) will be among the most important steps towards better reproduction of extreme temperature events in climate model simulations and more credible scenarios of their possible future changes.
The model data used in our study are available within the EURO-CORDEX project (https://euro-cordex.net/), E-OBS data are downloadable at https://www.ecad.eu/download/ensembles/download.php, and the ERA-INTERIM data are available in the ECMWF database at http://apps.ecmwf.int/datasets/.
A preliminary version of this work was presented as a poster at the EGU 2018 conference.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
The study was supported by the Czech Science Foundation (project 16-22000S). We acknowledge the World Climate Research Programme’s Working Group on Regional Climate and the Working Group on Coupled Modelling, former coordinating body of CORDEX and responsible panel for CMIP5. We also thank the climate modelling groups (listed in Table 1 of this paper) for producing and making available their model output. We also acknowledge the Earth System Grid Federation infrastructure an international effort led by the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison, the European Network for Earth System Modelling, and other partners in the Global Organisation for Earth System Science Portals (GO-ESSP). The E-OBS dataset was developed within the EU-FP6 ENSEMBLES project and is provided by the ECA&D project.
A. De Bono, G. Giuliani, S. Kluser, and P. Peduzzi, “Impacts of summer 2003 heat wave in Europe,” UNEP/DEWA/GRID Environment Alert Bulletin, vol. 2, pp. 1–4, 2004.View at: Google Scholar
IPCC, Climate Change 2014: Synthesis Report. Contribution of Working Groups I, II and III to the Fifth Assessment Report of the Intergovernmental Panel on Climate, IPCC, Geneva, Switzerland, 2014.
P. Van der Linden and J. F. B. Mitchell, ENSEMBLES: Climate Change and Its Impacts: Summary of Research and Results from the ENSEMBLES Project, Met Office Hadley Centre, London, UK, 2009.
M. R. Haylock, N. Hofstra, A. M. G. Klein Tank, E. J. Klok, P. D. Jones, and M. New, “A European daily high-resolution gridded data set of surface temperature and precipitation for 1950–2006,” Journal of Geophysical Research, vol. 113, no. D20, p. D20119, 2008.View at: Publisher Site | Google Scholar
A. F. Jenkinson and F. P. Collison, “An initial climatology of gales over the North Sea,” Synoptic Climatology Branch Memorandum No. 62. Meteorological Office, Bracknell, England, 1977.View at: Google Scholar
B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap, Chapman & Hall, New York, NY, USA, 1993.