- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents

Advances in Meteorology

Volume 2013 (2013), Article ID 731647, 15 pages

http://dx.doi.org/10.1155/2013/731647

## The Impact of Mesoscale Environmental Uncertainty on the Prediction of a Tornadic Supercell Storm Using Ensemble Data Assimilation Approach

^{1}Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, Norman, OK 73072, USA^{2}NOAA/OAR/National Severe Storms Laboratory, National Weather Center, 120 David L. Boren Boulevard, Norman, OK 73072, USA^{3}Center for Analysis and Prediction of Storms, University of Oklahoma, Norman, OK 73072, USA

Received 10 May 2013; Revised 1 August 2013; Accepted 22 August 2013

Academic Editor: Ming Xue

Copyright © 2013 Nusrat Yussouf et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Numerical experiments over the past years indicate that incorporating environmental variability is crucial for successful very short-range convective-scale forecasts. To explore the impact of model physics on the creation of environmental variability and its uncertainty, combined mesoscale-convective scale data assimilation experiments are conducted for a tornadic supercell storm. Two 36-member WRF-ARW model-based mesoscale EAKF experiments are conducted to provide background environments using either fixed or multiple physics schemes across the ensemble members. Two 36-member convective-scale ensembles are initialized using background fields from either fixed physics or multiple physics mesoscale ensemble analyses. Radar observations from four operational WSR-88Ds are assimilated into convective-scale ensembles using ARPS model-based 3DVAR system and ensemble forecasts are launched. Results show that the ensemble with background fields from multiple physics ensemble provides more realistic forecasts of significant tornado parameter, dryline structure, and near surface variables than ensemble from fixed physics background fields. The probabilities of strong low-level updraft helicity from multiple physics ensemble correlate better with observed tornado and rotation tracks than probabilities from fixed physics ensemble. This suggests that incorporating physics diversity across the ensemble can be important to successful probabilistic convective-scale forecast of supercell thunderstorms, which is the main goal of NOAA’s Warn-on-Forecast initiative.

#### 1. Introduction

The development and evolution of severe thunderstorm events are strongly tied to the environment, and therefore incorporating mesoscale environmental variability and its uncertainty is crucial for successful convective-scale data assimilation and forecasts [1–3]. Several studies illustrate the importance of incorporating the influence of environmental variability and mesoscale forcing on the storm scale flows for accurate prediction of tornadic supercell thunderstorms ([4, 5]). In particular, when Stensrud and Gao [4] use a more realistic inhomogeneous mesoscale environment as initial and boundary conditions for their convective-scale three-dimensional variational (3DVAR) data assimilation and forecast system, substantial improvement in forecast accuracy is obtained over a similar convective-scale system using a homogeneous, single-sounding environment, which is typical of idealized storm modeling studies. Yussouf et al. [6] investigate the benefits of using a combined mesoscale-convective scale cycled ensemble data assimilation and prediction system to investigate the accuracy of a very short-range (0-1 h) ensemble forecast of a tornadic supercell storm. The same suite of physics parameterization schemes is applied to the members of the mesoscale ensemble, which is used to provide environmental initial and boundary conditions for a convective-scale ensemble system, and the results are very encouraging. The convective-scale ensemble captures the structure and propagation of the main supercell storm and predicts the probability of a strong low-level vorticity track for the tornadic supercell that correlates well with the observed rotation track.

However, while providing mesoscale environmental variability is critical to severe weather forecasts, model bias errors due to the uncertainties associated with the physical parameterization schemes are unavoidable and are a known problem in convective-scale forecasting [7, 8]. Romine et al. [9] show that using the same set of physical parameterization suites across mesoscale ensemble members leads to unique bias errors, and when these mesoscale ensembles are used as a background field for convective-scale model, the forecast skill and accuracy degrade. Due to our limited understanding of atmospheric processes, it is likely that the model physics parameterizations schemes will face challenges in some convective environments. Removing model biases in a data assimilation system is very difficult and is an active area of research [10]. One approach to account for the model biases due to its uncertainties associated with physics parameterizations schemes is to allow for the inclusion of multiple physical parameterization schemes amongst the ensemble members [11]. Fujita et al. [12] find that an ensemble with both physics and initial condition uncertainties shows considerable improvement in forecasts of storm environment with improved location and intensity of drylines, frontal boundaries, and planetary boundary layer height and structure. Since the quality of convective-scale analyses and forecasts is so sensitive to background environmental variability, using an ensemble that contains uncertainties in both initial and model physics parameterization schemes is important and can positively impact the forecasts of convective events.

To study the impact of environmental variability and its uncertainty in the forecasts of severe thunderstorm events, an ensemble-based mesoscale and convective-scale data-assimilation and prediction system is developed for May 8, 2003, Oklahoma City (OKC), Oklahoma (OK) tornadic supercell storm. The OKC tornado is one of the most destructive events that occurred during a multiday tornado outbreak across the central and eastern United States in early May 2003 [13] and several data assimilation and forecast studies have focused on this particular storm [6, 14–16]. Two 36-member ensemble data assimilation experiments are conducted at mesoscale resolution to provide background environments for convective-scale ensembles: a FixedPhysics ensemble with the same set of physics parameterization schemes amongst the members [6, 9] and a MultiPhysics ensemble with members having different physical parameterization schemes to account for model physics uncertainty [12, 17–19]. In addition, each member from the two ensemble systems has slightly perturbed initial conditions to account for uncertainties in the atmospheric state. These two mesoscale ensembles are used to provide the initial and boundary conditions for the convective-scale ensemble data assimilation system centered in OKC and covering parts of the surrounding states of Kansas, Missouri, Arkansas, and Texas.

The main objective of this study is to investigate the accuracy of a very short-range (0-1 h) ensemble forecast of the OKC tornadic storm due to two different ensemble depictions of storm environmental conditions. A brief overview of the OKC tornadic supercell thunderstorm event followed by the experiment design for both the mesoscale and convective-scale data assimilation systems is discussed in Section 2. Section 3 assesses the quantitative and qualitative results of the forecasts from the ensembles. A discussion of key results is found in Section 4.

#### 2. Experiment Design

##### 2.1. Overview of the Event

On May 8, 2003, a violent tornado passed through portions of Moore, a suburban city south of OKC, as well as the southeast OKC metropolitan area with F4 damage reported along its path. Prior to tornado formation in the mid- to late afternoon, the synoptic scale environment became increasingly favorable for severe tornadic thunderstorms [14, 20]. At around 2050 UTC, several small cells initiated along the dryline in west central Oklahoma with one of the cells maturing into an isolated supercell storm by 2130 UTC. Over the next hour, this supercell moved northeastward and intensified. A violent tornado developed around 2210 UTC and tracked east-northeastward for about 30 km until it dissipated at around 2238 UTC, leaving a damage path stretching from Moore to Choctaw, Oklahoma (Figure 1(b)). The National Weather Service (NWS) Office in Norman OK issued a tornado warning for the path of the storm, including Cleveland, McClain, and south Oklahoma counties at 2149 UTC, with approximately 21-minute lead time for Moore in Cleveland County and approximately 30-minute lead time for citizens in Oklahoma County.

##### 2.2. Mesoscale Ensembles and Cycled EnKF Data Assimilation System

The Advanced Research Weather Research and Forecasting (WRF-ARW core version 3.3.1; [21]) model is used to create the mesoscale ensemble data assimilation system. The model domain covers the continental United States (Figure 1(a)) with a horizontal grid spacing of 12 km and 51 vertical grid levels with vertically stretched grids from the surface to 50 hPa aloft. Two sets of 36-member ensembles are initialized at 1200 UTC May 8, 2003, using The National Centers for Environments Prediction’s (NCEP) Eta model for the ensemble mean initial and boundary conditions. Random samples of the horizontal components of wind, water vapor mixing ratio, and temperature are drawn from a default background error covariance file estimated by the NMC method [22] using the WRF data assimilation software. These samples are then added to each ensemble member to account for uncertainties in the initial and boundary conditions [23]. One experiment uses the same sets of physics parameterization schemes (FixedPhysics) across all 36 ensemble members. The physics options used are Thompson [24] for microphysics, Tiedtke [25, 26] for cumulus parameterization, YSU [27] for planetary boundary layer parameterization, RRTMG for both longwave and shortwave radiation, and Noah [28] for the land surface parameterization scheme. A second 36-member ensemble experiment uses different combinations of physics schemes (MultiPhysics) amongst the ensemble members to address the uncertainties in model physics parameterization schemes (e.g., [11, 12, 18, 19]). The diversity in physics options includes land surface, planetary boundary layer, radiation, convection, and microphysical parameterizations schemes and is shown in Table 1.

Both ensemble systems assimilate routinely available observations from NOAA’s Meteorological Assimilation Data Ingest System (MADIS) every hour starting at 1300 UTC May 8, 2003, and extending to 0000 UTC May 9, 2003 (Figure 1(a)), using the ensemble adjustment Kalman filter (EAKF; [29]) within the Data Assimilation Research Testbed (DART) software [30, 31].

A half radius of 230 km in the horizontal and a half radius of 4 km in the vertical are used for the covariance localization function (the fifth order correlation function from [32]). The observations assimilated in the ensembles are the surface altimeter setting, pressure, temperature, dewpoint, and horizontal wind components from land and marine surface stations, rawinsondes, and aircraft. The predicted variables updated by the data assimilation scheme include the three wind components, perturbation temperature, perturbation geopotential, perturbation surface pressure of dry air, potential temperature tendency due to microphysics, water vapor, and hydrometeors. Also updated are the 10 m wind fields, 2 m temperature and water vapor fields, and total surface pressure variables, which are diagnosed by the surface and boundary layer schemes using state variables on the model grid. The FixedPhysics and MultiPhysics mesoscale ensemble analyses are then used to create the initial background and boundary conditions for their associated convective-scale ensembles.

##### 2.3. Convective-Scale Ensembles and Cycled 3DVAR Data Assimilation System

The model used for the two convective-scale ensemble data assimilation and forecasts experiments is the Advanced Regional Prediction System (ARPS; [33, 34]) and its 3DVAR [4, 14, 35–37] and cloud analyses scheme [14, 38]. The ARPS 3DVAR system has been successfully used in NOAA’s Hazardous Weather Testbed (HWT) Spring Forecast experiments [39–41] for the past several years to analyze and detect convective-scale severe weather events [42]. Two 36-member convective-scale ensembles are initialized from the FixedPhysics and MultiPhysics mesoscale ensemble analyses at 2100 UTC. Thus, the mesoscale ensembles provide environmental background fields and boundary conditions for their associated convective-scale 3DVAR data assimilation system. The convective-scale domain is centered in OKC using 3 km horizontal grid spacing with 192 × 192 × 50 grid points and is selected such that sufficient distance is maintained between the supercell storm and lateral boundaries (Figure 1(b)). Radar observations are assimilated into each of the individual convective-scale ensemble members using the 3DVAR system. The convective-scale ensembles are referred to as FixedPhysics and MultiPhysics in reference to the mesoscale ensemble system that provides the initial and boundary conditions. The physics options used for both FixedPhysics and MultiPhysics convective-scale ensembles are identical and include Lin et al. [43] for microphysics, Noah [28] for land surface, Mellor-Yamada-Janjic (MYJ; [44, 45]) for planetary boundary layer, Dudhia [46] for shortwave, and RRTM [47] for longwave radiation parameterization schemes. Cumulus parameterization is turned off for the convective-scale ensemble. The only differences in the two convective-scale ensembles result from the use of different mesoscale environmental conditions provided by either the FixedPhysics or MultiPhysics 12 km mesoscale ensembles.

Reflectivity and radial velocity observations from four operational Weather Surveillance Radar-1988 Doppler (WSR-88D) radars located at Vance Air Force Base (KVNX), Twin Lakes (KTLX), Tulsa (KINX), and Frederick (KFDR) are assimilated into the two convective-scale ensembles (Figure 1(b)). The radar observations are processed using the 88D2ARPS software with the necessary quality control steps, including velocity dealiasing and ground clutter removal [48]. The quality controlled radar observations are then projected into the model grid space in the form of a series of column observations. In order to mitigate the negative impact of small spurious cells, the noisy data in the radar observations are discarded if the reflectivity is smaller than 25 dBZ. The latent heat (LH) release based method from the ARPS cloud analysis package is used for in-cloud temperature adjustment and all hydrometeor variables are updated during every analysis in the assimilating window. The ARPS 3DVAR uses the radar radial velocity and Oklahoma Mesonet [49] surface observations of temperature, pressure, wind speed and direction, and dewpoint temperature to update the three wind components (, , and ), potential temperature (), pressure (), and water vapor mixing ratio (), while the cloud analysis procedure uses the reflectivity observations to update the hydrometeor variables and adjust the in-cloud temperature and moisture fields. Additional quality control of the radial velocity observations is conducted during ARPS 3DVAR assimilation, such that if the absolute difference between a gridded radial velocity and the background is too high (greater than 20 m s^{−1}), that observation is rejected. Radar observations are assimilated into each member of the FixedPhysics and MultiPhysics ensemble members via a 5 min cycling procedure that lasts for a 40 min period starting at 2100 UTC and ending at 2140 UTC with a total of nine assimilation cycles. Each cycle begins with an application of the 3DVAR and cloud analysis, followed by a 5 min ARPS forecast, which is then used as the background for the next 3DVAR and cloud analysis. One-hour ensemble forecasts are launched from each of the 36 convective-scale ensemble analyses valid at the end of the cycling period at 2140 UTC. This time is 30 min prior to the time the OKC tornado first developed in the city of Moore, Oklahoma.

#### 3. Results

The accuracy of the forecasts from both mesoscale and convective scale experiments using either FixedPhysics or MultiPhysics ensemble is evaluated using both quantitative and qualitative perspectives. Statistical measures include root-mean-square (RMSE) error, bias (forecast observations), and equitable threat scores (ETS) [50]. The environmental soundings, dryline structures, significant tornado parameter (STP), and forecast probability of low-level updraft helicity track from the two ensemble systems are also compared to quantify the accuracy of the storm forecasts using two different inhomogeneous mesoscale storm environments.

##### 3.1. Environmental Soundings from the Mesoscale Ensembles

Soundings from the mesoscale ensembles at 2100 UTC May 8, and 0000 UTC May 9, 2003, from Oklahoma City (KOKC) indicate that the two ensembles produce different storm environments (Figure 2). The environmental soundings at 2100 UTC, the time when convective-scale ensembles are initialized from the two mesoscale ensembles, show noticeable differences between the two ensemble systems (Figures 2(a) and 2(b)). The soundings from the MultiPhysics ensemble that incorporates physics parameterizations diversity across the members show larger variability amongst the members than those from the FixedPhysics ensemble with the same single suite of parameterization schemes among the members. All 36 ensemble members from the FixedPhysics ensemble show saturated air around 850 hPa while the MultiPhysics ensemble members show greater variability in temperature and humidity from the surface to 700 hPa. The winds in the lowest 3 km are also more variable in MultiPhysics, with stronger backing of the surface winds in MultiPhysics. Due to the lack of radiosonde observations at 2100 UTC, it is not known which soundings are more realistic. Soundings from the two ensemble systems later in the evening at 0000 UTC show that both ensemble systems fail to capture the observed capping inversion (Figures 2(c) and 2(d)). Accurately capturing the capping inversion is a common forecasting problem faced by the modeling community. However, the larger variability within MultiPhysics captures the observed temperature and moisture profiles within the member envelope for most vertical levels, an improvement over that seen from FixedPhysics. The observations more often lie on the edge or outside the ensemble envelope for the FixedPhysics experiment.

##### 3.2. Location of Drylines in the Convective-Scale Ensembles

The forecast locations of the dryline—the feature that helped initiate the OKC supercell storm—and their associated dryline bulges also are important to compare between the two ensembles. Isolines of 10°C 2 m dewpoint temperature forecasts (a reasonable proxy for dryline location) from each member of FixedPhysics and MultiPhysics along with the analyzed isoline from Oklahoma Mesonet observations are shown in Figure 3. The MultiPhysics ensemble has dryline bulges (areas where dry air is advancing eastward more rapidly yielding an eastward bulge in the isodrosotherm) in Oklahoma as early as 10 min into the forecasts at 2150 UTC, in reasonable agreement with observations, while the FixedPhysics ensemble has no dryline bulges at this time (Figures 3(a)–3(c)). The MultiPhysics ensemble captures the dryline location within the ensemble envelope better than the FixedPhysics ensemble throughout the 1 h forecast (Figures 3(d)–3(l)). Most importantly, the MultiPhysics ensemble also produces two distinct dryline bulges in Oklahoma that compare well with the two observed dryline bulges. Dryline bulges are an indication of the development of deep moist convection and they develop due to enhanced low-level convergence, helping parcels reach their level of free convection [51].

##### 3.3. Forecast Error Statistics of Near Surface Variables

Bias and RMSE of 2 m temperature, 2 m dewpoint temperature, and 10 m wind speed are calculated from the two convective-scale ensembles and corresponding Oklahoma Mesonet observations at 5 min intervals using the 112 available Mesonet observations stations within the model domain (Figure 4). The RMSE from the MultiPhysics ensemble is smaller compared to the FixedPhysics ensemble throughout the entire forecast period for both 2 m temperature and 2 m dewpoint temperature (Figures 4(a) and 4(b)). The differences in the magnitude of the RMSE errors are as high as 0.35°C and 0.60°C for 2 m temperature and 2 m dewpoint temperature, respectively, at the beginning of the forecast period, with the differences reducing to 0.08 and 0.10, respectively, at the end of forecast period. The differences in the RMSE values between the two ensembles for the 10 m wind speed are very small with slightly smaller values for the MultiPhysics ensemble (Figure 4(c)). These results are consistent with the findings from Fujita et al. [12] and Zhiyong and Zhang [17] in which the benefits of a MultiPhysics ensemble over a single-scheme ensemble are found to be more pronounced in the thermodynamic variables than in the wind fields. For 2 m temperature, the FixedPhysics ensemble has a larger warm bias while the MultiPhysics ensemble has a smaller cold bias at all forecast times. For 2 m dewpoint temperature both FixedPhysics and MultiPhysics ensembles have a moist bias with the bias from MultiPhysics being larger. However, for the 10 m wind speed, the bias in MixedPhysics is consistently larger than that in FixedPhysics. These statistics indicate that using physics diversity across the ensemble can have a positive impact on the forecast of near surface thermodynamic variables but a mixed impact on the forecast of near surface wind field.

##### 3.4. Ensemble Mean Forecasts of Significant Tornado Parameter (STP)

One of the severe weather parameters used to evaluate tornadic supercell environments by the NOAA/NWS/Storm Prediction Center is the significant tornado parameter (STP; [52]). The STP helps discriminate between significantly tornadic (F2 or greater damage) and nontornadic supercell environments, with proximity soundings yielding STP values greater than 1 in association with a majority of F2 or greater tornadic supercell storms. The STP equation is defined as where CAPE is the convective available potential energy, SHR is 0–6 km vector vertical shear magnitude, SREH is 0-1 km storm-relative helicity, CIN is convective inhibition, and LCL is the lifting condensation level. The ensemble-mean forecast of STP derived from MultiPhysics ensemble at 2150 (20 minutes prior to tornadogenesis) is very large around the OKC area, with values approaching 50, suggestive of a severe storm environment with significant tornado threat (Figure 5(b)). Thompson et al. [52] show that the largest values of STP are below 10 when using proximity soundings from the hourly 40 km Rapid Update Cycle-2 (RUC-2), suggesting that the high temporal frequency 3 km convective-scale model forecasts over an area that includes both the supercell storm and its surrounding environments may be providing new and useful information. The maximum value of STP continues to increase over the next 20 minutes out to 2200 UTC (Figure 5(d)) with values higher than 75. By the time the observed tornado forms at ~2210 UTC, the values of STP start to decrease (Figures 5(f) and 5(h)). In contrast, the FixedPhysics ensemble generates smaller STP values around OKC at 2150 UTC indicating a less favorable storm environment (Figure 5(a)). By 2200 UTC, the FixedPhysics ensemble produces high values of STP in south central Kansas (Figure 5(c)) indicating severe tornadic environment in that area and small values of STP around OKC area. Thus, the STP values from the FixedPhysics ensemble could have diverted forecasters attention to the north of Kansas where no significant tornadoes were observed until over 30 min after the end of the forecast period. The behavior of the large magnitude STP fields to the south of regions of forecast convection suggests that they are produced by the model supercells modifying the surrounding environment as also seen in Brooks et al. [53]. The correlation coefficient between the maximum values of STP surrounding the supercell region and the maximum values of 0–3 km updraft helicity within the storm (a measure of low-level storm intensity) during the forecast period is 0.86 for FixedPhysics ensemble and 0.95 for MultiPhysics ensemble. These high correlations suggest that the intensity of the environmental modification is related to the intensity of the low-level mesocyclone. This relationship deserves further study to evaluate whether or not it could be used to evaluate the likelihood of tornado formation.

##### 3.5. Ensemble Probabilistic Forecast of Updraft Helicity of the Supercell

The 3 km model horizontal grid spacing used in this study is far too coarse to explicitly resolve a tornado circulation. However, one good measure of the amount of rotation within the supercell storm is the updraft helicity (UH; [39–41, 54]), as it tends to highlight the main rotating storm updraft within a specified layer. A 0–3 km UH is selected to evaluate forecasts of low-level mesocyclones associated with tornadic supercell storms. Neighborhood ensemble probabilities of UH exceeding predetermined thresholds are calculated during the 1 h forecast period from both experiments (Figures 6 and 7), with a 9 km radius used to calculate the probabilistic forecasts of UH around each horizontal grid point to account for the small displacement errors across the ensemble members [6]. Results using a threshold UH of 50 m^{2} s^{−2}, a reasonable value for identifying mesocyclonic features in a convective-scale model [55, 56], show several regions of interest (Figure 6). Both experiments show maximum probabilities (100%) of significant rotation over OKC that covers the NWS surveyed OKC tornado observed damage track (black line) and extends farther northeastward. Close examination reveals that the 100% probabilities of a low-level mesocyclone from the FixedPhysics ensemble (Figure 6(a)) encompass a broader area than those from the MultiPhysics ensemble (Figure 6(b)) for the OKC area. In addition, both experiments show several additional rotation tracks north of OKC in north central Oklahoma, near the Oklahoma-Kansas border and in south-central Kansas. The FixedPhysics ensemble experiments show two high probability rotation tracks, one just north of OKC with 100% probabilities at several points and another in south-central Kansas with probabilities as high as 95%. In contrast, the MultiPhysics ensemble generates low probabilities of rotation with values below 45% on the storm north of OKC and values below 70% for the longer mesocyclone track in south Kansas. The high probabilities of UH in Kansas from the FixedPhysics ensembles correlate with the high STP values in that area as shown in Figures 5(a), 5(c), 5(e), and 5(g). These results suggest that compared to the FixedPhysics ensemble, the MultiPhysics ensemble is able to better discriminate the region of tornadic supercell threat during this 1-hour forecast period.

Using UH track as a proxy for tornado path length forecasts, Clark et al. [41] show that the UH forecast path length from convective-scale models is strongly related to the track length of observed tornadoes. Therefore, to evaluate the forecasts of low-level tornadic rotation for the OKC supercell storm, 0–3 km neighborhood UH probabilities with higher threshold values of 150 m^{2} s^{−2}, 200 m^{2} s^{−2}, and 250 m^{2} s^{−2} from both FixedPhysics and MultiPhysics convective-scale ensemble are evaluated (Figures 7(a)–7(f)) and are compared against the 0–3 km mesocyclone circulations [57] from KTLX radar observations (Figure 7(g)) generated using the Warning Decision Support System-Integrated Information software (WDSS-II; [58]). Results indicate that the low-level mesocyclone persists during the 0-1 h forecast for all threshold values, with higher probabilities of UH qualitatively correlating well with the observed rotation track (Figure 7(g)). Maximum probabilities (100%) are seen at all grid points covering the NWS damage path and correlating well with the radar observed rotation path for 150 m^{2} s^{−2} threshold value for both ensemble experiments (Figures 7(a), 7(b), and 7(g)). However, the UH track from FixedPhysics (Figure 7(a)) extends well beyond the observed mesocyclone track with 100% probabilities stretching northeastward, while the MultiPhysics (Figure 7(b)) ensemble correctly forecasts the length of observed rotation with 100% probabilities and has lower UH probabilities beyond the observed rotation track. The probabilities remain above 90% for a UH threshold of 200 m^{2} s^{−2} and above 50% for a threshold of 250 m^{2} s^{−2} for the entire path length of the observed damage track (Figures 7(d) and 7(f)) in MultiPhysics. In contrast, the FixedPhysics ensemble indicates lower UH probabilities with values below 65% for a 200 m^{2} s^{−2} threshold and below 30% for a 250 m^{2} s^{−2} threshold near the beginning of the observed tornado. Thus, the UH probability track from the MultiPhysics ensemble better captures the observed tornado and rotation track extent than from the FixedPhysics ensemble. These results highlight the potential benefit of background environmental variability in predicted 0–3 km UH forecast probabilities violent tornadoes, one of the goals of NOAA’s Warn-on-Forecast initiative [18].

##### 3.6. Forecast Time Series of Equitable Threat Scores (ETS)

To quantify the accuracy of precipitation forecasts from the ensembles, the ETS is calculated from both FixedPhysics and MultiPhysics convective-scale ensembles for radar reflectivity exceeding threshold values of 35 and 45 dBZ (Figure 8). The ETS is calculated using continuously cycled 3DVAR analyses produced throughout the 1 h forecast period as observations. An ETS score of 1 indicates a perfect forecast, with the ETS value decreasing to 0 as forecast accuracy declines. Results indicate that both ensemble systems start with ETS values of ~0.70 for 35 dBZ threshold (Figures 8(a) and 8(b)) and ~0.55 for 45 dBZ threshold (Figures 8(c) and 8(d)) at the beginning of the forecast. The ETS accuracy decreases with forecast lead times as expected. However, the variability in the ETS score amongst the members is larger and increases with forecast lead times for the MultiPhysics ensembles compared to that for the FixedPhysics ensemble. At the end of the forecast period at 2240 UTC, the mean ETS values for MultiPhysics ensemble are ~0.15 and ~0.20 (Figures 8(b) and 8(d)), while the mean ETS values for FixedPhysics ensembles are 0.09 and 0.10 (Figures 8(a) and 8(c)) for 35 and 45 dBZ thresholds, respectively. Thus the MultiPhysics ensemble maintains higher ETS accuracy than the FixedPhysics ensemble at the end of 1-hour-long forecasts. This is more pronounced for 45 dBZ threshold (Figure 8(d)), in which the MultiPhysics ensemble maintains the 0.20 ETS values during the last 25 minutes of the forecasts.

#### 4. Discussion

In this study, experiments are conducted to assimilate radar observations within a convective-scale ensemble using background storm environments from two different mesoscale ensembles for May 8, 2003, Oklahoma City tornadic supercell storm event. The two sets of 36-member 12 km mesoscale ensembles using either single (FixedPhysics) or multiple physical parameterization (MultiPhysics) schemes are produced. The FixedPhysics ensemble uses the same land surface, planetary boundary layer, radiation, convection, and microphysical parameterizations amongst all the ensemble members, whereas the MultiPhysics ensemble uses a variation of those combinations across the members. Traditional atmospheric observations are assimilated into the ensembles at every hour cycle starting at 1200 UTC on the day of the event and out to 12 h or 0000 UTC, May 9, 2003. The convective-scale 3 km ensembles are created using the mesoscale ensembles as background and assimilating Doppler radial velocity and reflectivity observations from four operational WSR-88D radars every 5 minutes over a 40 min cycling period starting at 2100 UTC and ending at 2140 UTC. Finally, 1 h forecasts are launched from the convective-scale ensemble analyses starting at 2140 UTC and extending out to 2240 UTC, thereby covering the entire lifetime of the observed OKC tornado.

Results indicate that the forecast RMSE values for the near surface temperature, dewpoint temperature, and wind variables from the convective-scale MultiPhysics ensemble are smaller than those from the FixedPhysics ensemble, highlighting the positive impact of the MultiPhysics approach. However, a more qualitative evaluation of specific forecast features, such as the presence of dryline bulges, environmental sounding structures, values of ensemble mean STP, and 0–3 km UH probabilities shows that the MultiPhysics ensemble better captures the important features on this day than the FixedPhysics ensemble. In particular, the convective-scale Multiphysics ensemble forecasts high values of STP around the OKC area before tornadogenesis, suggesting an environment that is very favorable for tornadic supercell storms, while the FixedPhysics experiment forecasts much lower STP values in the same area. The 0–3 km UH values for both FixedPhysics and MultiPhysics ensembles show high probabilities that correlate well with the observed tornado and low-level rotation tracks. However, the UH track in the MultiPhysics ensemble better captures the beginning and ending points of the observed tornado track than seen in the FixedPhysics ensemble. Therefore, convective-scale ensembles with greater diversity in the mesoscale environmental conditions as produced through using multiple physics schemes can provide forecasters with more accurate situational awareness and greater confidence of the tornado threats from very short-range ensemble forecasts.

Although not computationally feasible for this study, convective-scale data assimilation and forecast experiments with horizontal grid spacing less than 1 km are needed to resolve tornadic-scale circulations. Past studies show noticeable differences in storm structures when simulated with a horizontal grid spacing varying between 250 m and 1 km [59, 60]. While the computational demands associated with such small grid spacing are significant at this time, with continued rapid increases in computing power, future work will focus on convective-scale data assimilation and forecast experiments at 1 km or less. The use of more sophisticated double or triple moment microphysics schemes in the convective-scale ensemble with perturbed microphysical parameters within the scheme [61] and applying physics diversity across the ensemble [62, 63] can provide improved short-range forecasts for a wide range of storm systems and will be included in future convective-scale data assimilation studies.

Due to our limited understanding of atmospheric processes, it is likely that the use of even more sophisticated physical parameterization schemes will face challenges when used in some storm environments. However, the results obtained from this study suggest that by using reasonable diversity in physics schemes, an ensemble system is more likely to span the observations and provide improved storm environments for a wide range of storm systems. An ensemble system that accounts for uncertainties both in initial condition and model physical parameterization schemes is important to the successful very short-range probabilistic convective-scale forecast of tornadic supercell thunderstorms, which is the main goal NOAA’s Warn-on-Forecast initiative.

#### Acknowledgments

The constructive comments of two anonymous reviewers greatly improved the paper. Local computer assistance provided by Brett Morrow, Steven Fletcher, Brad Swagowitz, and Karen Cooper is greatly appreciated. Partial funding for this research was provided by NOAA/Office of Oceanic and Atmospheric Research under NOAA-University of Oklahoma Cooperative Agreement NA17RJ1227, U.S. Department of Commerce.

#### References

- A. Aksoy, D. C. Dowell, and C. Snyder, “A multicase comparative assessment of the ensemble Kalman filter for assimilation of radar observations. Part I: storm-scale analyses,”
*Monthly Weather Review*, vol. 137, no. 6, pp. 1805–1824, 2009. View at Publisher · View at Google Scholar · View at Scopus - A. Aksoy, D. C. Dowell, and C. Snyder, “A multicase comparative assessment of the ensemble Kalman filter for assimilation of radar observations. Part II: short-range ensemble forecasts,”
*Monthly Weather Review*, vol. 138, no. 4, pp. 1273–1292, 2010. View at Publisher · View at Google Scholar · View at Scopus - R. M. Cintineo and D. J. Stensrud, “On the predictability of supercell thunderstorm evolution,”
*Journal of Atmospheric Science*, vol. 70, pp. 1993–2011, 2013. View at Google Scholar - D. J. Stensrud and J. Gao, “Importance of horizontally inhomogeneous environmental initial conditions to ensemble storm-scale radar data assimilation and very short-range forecasts,”
*Monthly Weather Review*, vol. 138, no. 4, pp. 1250–1272, 2010. View at Publisher · View at Google Scholar · View at Scopus - C. L. Ziegler, E. R. Mansell, J. M. Straka, D. R. Macgorman, and D. W. Burgess, “The impact of spatial variations of low-level stability on the life cycle of a simulated supercell storm,”
*Monthly Weather Review*, vol. 138, no. 5, pp. 1738–1766, 2010. View at Publisher · View at Google Scholar · View at Scopus - N. Yussouf, E. R. Mansell, L. J. Wicker, D. M. Wheatley, and D. J. Stensrud, “The ensemble Kalman filter analyses and forecasts of the 8 May 2003 Oklahoma City Tornadic Supercell Storm using single and double moment microphysics schemes,”
*Monthly Weather Review*, 2013. View at Publisher · View at Google Scholar - C. S. Schwartz, J. S. Kain, S. J. Weiss et al., “Next-day convection-allowing WRF model guidance: a second look at 2-km versus 4-km grid spacing,”
*Monthly Weather Review*, vol. 137, no. 10, pp. 3351–3372, 2009. View at Publisher · View at Google Scholar · View at Scopus - C. S. Schwartz, J. S. Kain, S. J. Weiss et al., “Toward improved convection-allowing ensembles: model physics sensitivities and optimizing probabilistic guidance with small ensemble membership,”
*Weather and Forecasting*, vol. 25, no. 1, pp. 263–280, 2010. View at Publisher · View at Google Scholar · View at Scopus - G. S. Romine, C. S. Schwartz, C. Snyder, J. L. Anderson, and M. L. Weisman, “Model bias in a continuously cycled assimilation system and its influence on convection-permitting forecasts,”
*Monthly Weather Review*, vol. 141, pp. 1263–1284, 2013. View at Google Scholar - B. C. Ancell, “Examination of analysis and forecast errors of high-resolution assimilation, Bias removal, and digital filter initialization with an ensemble Kalman filter,”
*Monthly Weather Review*, vol. 140, pp. 3992–4004, 2012. View at Google Scholar - D. J. Stensrud, J.-W. Bao, and T. T. Warner, “Using initial condition and model physics perturbations in short-range ensemble simulations of mesoscale convective systems,”
*Monthly Weather Review*, vol. 128, no. 7 I, pp. 2077–2107, 2000. View at Google Scholar · View at Scopus - T. Fujita, D. J. Stensrud, and D. C. Dowell, “Surface data assimilation using an ensemble Kalman filter approach with initial condition and model physics uncertainties,”
*Monthly Weather Review*, vol. 135, no. 5, pp. 1846–1868, 2007. View at Publisher · View at Google Scholar · View at Scopus - T. M. Hamill, R. S. Schneider, H. E. Brooks et al., “The May 2003 extended tornado outbreak,”
*Bulletin of the American Meteorological Society*, vol. 86, no. 4, pp. 531–542, 2005. View at Publisher · View at Google Scholar · View at Scopus - M. Hu and M. Xue, “Impact of configurations of rapid intermittent assimilation of WSR-88D radar data for the 8 May 2003 Oklahoma City tornadic thunderstorm case,”
*Monthly Weather Review*, vol. 135, no. 2, pp. 507–525, 2007. View at Publisher · View at Google Scholar · View at Scopus - T. M. Lei, M. Xue, and T. Yu, “Multi-scale analysis and prediction of the 8 May 2003 Oklahoma City tornadic supercell storm assimilating radar and surface network data using EnKF,” in
*Proceedings of the 13th Conference on Integrated Observing and Assimilation Systems for Atmosphere, Oceans, and Land Surface (IOAS-AOLS '09)*, The American Meteorological Society, 2009. - D. C. Dowell, L. J. Wicker, and C. Snyder, “Ensemble kalman filter assimilation of radar observations of the 8 may 2003 oklahoma city supercell: influences of reflectivity observations on storm-scale analyses,”
*Monthly Weather Review*, vol. 139, no. 1, pp. 272–294, 2011. View at Publisher · View at Google Scholar · View at Scopus - Z. Meng and F. Zhang, “Tests of an ensemble Kalman filter for mesoscale and regional-scale data assimilation. Part IV: comparison with 3DVAR in a month-long experiment,”
*Monthly Weather Review*, vol. 136, no. 10, pp. 3671–3682, 2008. View at Publisher · View at Google Scholar · View at Scopus - D. J. Stensrud, N. Yussouf, D. C. Dowell, and M. C. Coniglio, “Assimilating surface data into a mesoscale model ensemble: cold pool analyses from spring 2007,”
*Atmospheric Research*, vol. 93, no. 1–3, pp. 207–220, 2009. View at Publisher · View at Google Scholar · View at Scopus - D. M. Wheatley, D. J. Stensrud, D. C. Dowell, and N. Yussouf, “Application of a WRF mesoscale data assimilation system to springtime severe weather events 2007–09,”
*Monthly Weather Review*, vol. 140, pp. 1539–1557, 2012. View at Google Scholar - G. S. Romine, D. W. Burgess, and R. B. Wilhelmson, “A dual-polarization-radar-based assessment of the 8 May 2003 Oklahoma City Area tornadic supercell,”
*Monthly Weather Review*, vol. 136, no. 8, pp. 2849–2870, 2008. View at Publisher · View at Google Scholar · View at Scopus - W. C. Skamarock, J. B. Klemp, J. Dudhia et al., “A description of the Advanced Research WRF version 2,” Tech. Rep., NCAR, TN-468+STR, 2005. View at Google Scholar
- D. F. Parrish and J. C. Derber, “The National Meteorological Center's spectral statistical- interpolation analysis system,”
*Monthly Weather Review*, vol. 120, no. 8, pp. 1747–1763, 1992. View at Google Scholar · View at Scopus - R. D. Torn, G. J. Hakim, and C. Snyder, “Boundary conditions for limited-area ensemble Kalman filters,”
*Monthly Weather Review*, vol. 134, no. 9, pp. 2490–2502, 2006. View at Publisher · View at Google Scholar · View at Scopus - G. Thompson, P. R. Field, R. M. Rasmussen, and W. D. Hall, “Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part II: implementation of a new snow parameterization,”
*Monthly Weather Review*, vol. 136, no. 12, pp. 5095–5115, 2008. View at Publisher · View at Google Scholar · View at Scopus - M. Tiedtke, “A comprehensive mass flux scheme for cumulus parameterization in large-scale models,”
*Monthly Weather Review*, vol. 117, no. 8, pp. 1779–1800, 1989. View at Google Scholar · View at Scopus - C. Zhang, Y. Wang, and K. Hamilton, “Improved representation of boundary layer clouds over the southeast pacific in ARW-WRF using a modified tiedtke cumulus parameterization scheme,”
*Monthly Weather Review*, vol. 139, no. 11, pp. 3489–3513, 2011. View at Publisher · View at Google Scholar · View at Scopus - S.-Y. Hong, Y. Noh, and J. Dudhia, “A new vertical diffusion package with an explicit treatment of entrainment processes,”
*Monthly Weather Review*, vol. 134, no. 9, pp. 2318–2341, 2006. View at Publisher · View at Google Scholar · View at Scopus - F. Chen and J. Dudhia, “Coupling and advanced land surface-hydrology model with the Penn State-NCAR MM5 modeling system. Part I: model implementation and sensitivity,”
*Monthly Weather Review*, vol. 129, no. 4, pp. 569–585, 2001. View at Google Scholar · View at Scopus - J. L. Anderson, “An ensemble adjustment Kalman filter for data assimilation,”
*Monthly Weather Review*, vol. 129, no. 12, pp. 2884–2903, 2001. View at Google Scholar · View at Scopus - J. L. Anderson and N. Collins, “Scalable implementations of ensemble filter algorithms for data assimilation,”
*Journal of Atmospheric and Oceanic Technology*, vol. 24, no. 8, pp. 1452–1463, 2007. View at Publisher · View at Google Scholar · View at Scopus - J. Anderson, T. Hoar, K. Raeder et al., “The data assimilation research testbed a community facility,”
*Bulletin of the American Meteorological Society*, vol. 90, no. 9, pp. 1283–1296, 2009. View at Publisher · View at Google Scholar · View at Scopus - G. Gaspari and S. E. Cohn, “Construction of correlation functions in two and three dimensions,”
*Quarterly Journal of the Royal Meteorological Society*, vol. 125, no. 554, pp. 723–757, 1999. View at Google Scholar · View at Scopus - M. Xue, K. K. Droegemeier, and V. Wong, “The Advanced Regional Prediction System (ARPS)—a multi-scale nonhydrostatic atmospheric simulation and prediction model. Part I: model dynamics and verification,”
*Meteorology and Atmospheric Physics*, vol. 75, no. 3-4, pp. 161–193, 2000. View at Google Scholar · View at Scopus - M. Xue, K. K. Droegemeier, V. Wong et al., “The Advanced Regional Prediction System (ARPS)—a multi-scale nonhydrostatic atmospheric simulation and prediction tool. Part II: model physics and applications,”
*Meteorology and Atmospheric Physics*, vol. 76, no. 3-4, pp. 143–165, 2001. View at Google Scholar · View at Scopus - J. Gao, M. Xue, A. Shapiro, and K. K. Droegemeier, “A variational method for the analysis of three-dimensional wind fields from two Doppler radars,”
*Monthly Weather Review*, vol. 127, no. 9, pp. 2128–2142, 1999. View at Google Scholar · View at Scopus - J. Gao, M. Xue, K. Brewster, F. Carr, and K. K. Droegemeier, “New development of a 3DVAR system for a nonhydrostatic NWP model,” in
*Proceedings of the 15th Conference of Numical Weather Prediction and 19th Conference of Weather Analyses and Forecasting*, pp. 339–341, The American Meteorological Society, San Antonio, Tex, USA, 2002. - J. Gao, M. Xue, K. Brewster, and K. K. Droegemeier, “A three-dimensional variational data analysis method with recursive filter for Doppler radars,”
*Journal of Atmospheric and Oceanic Technology*, vol. 21, no. 3, pp. 457–469, 2004. View at Google Scholar · View at Scopus - K. A. Brewster, “Recent advances in diabatic initialization of a non-hydrostatic numerical model,” in
*Proceedings of the 15th Conference on Numerical Weather Prediction and 21st Conference on Severe Local Storms*, American Meteorological Society, San Antonio, Tex, USA, 2001. - A. J. Clark, S. J. Weiss, J. S. Kain et al., “An overview of the 2010 Hazardous Weather Testbed experimental forecast program spring experiment,”
*Bulletin of the American Meteorological Society*, vol. 93, no. 1, pp. 55–74, 2012. View at Publisher · View at Google Scholar · View at Scopus - A. J. Clark, J. S. Kain, P. T. Marsh, J. Correia, M. Xue, and F. Kong, “Forecasting tornado pathlengths using a three-dimensional object identification algorithm applied to convection-allowing forecasts,”
*Weather and Forecasting*, vol. 27, pp. 1090–1113, 2012. View at Google Scholar - A. J. Clark, J. Gao, P. T. Marsh et al., “Tornado path length forecasts from 2010-2011 using ensemble updraft helicity,”
*Weather and Forecasting*, vol. 28, pp. 387–407, 2013. View at Google Scholar - J. Gao, T. T. Smith, and D. J. Stensrud, “A realtime weather-adaptive 3DVAR analysis system for severe weather detections and warnings,”
*Weather and Forecasting*, vol. 28, pp. 727–745, 2013. View at Google Scholar - Y.-L. Lin, R. D. Farley, and H. D. Orville, “Bulk parameterization of the snow field in a cloud model,”
*Journal of Climate & Applied Meteorology*, vol. 22, no. 6, pp. 1065–1092, 1983. View at Google Scholar · View at Scopus - G. L. Mellor and T. Yamada, “Development of a turbulence closure model for geophysical fluid problems,”
*Reviews of Geophysics & Space Physics*, vol. 20, no. 4, pp. 851–875, 1982. View at Google Scholar · View at Scopus - Z. I. Janjic, “Nonsingular implementation of the Mellor–Yamada Level 2.5 Scheme in the NCEP Meso model,”
*NCEP Office Note*437, 61, 2002. View at Google Scholar - J. Dudhia, “Numerical study of convection observed during the Winter Monsoon Experiment using a mesoscale two-dimensional model,”
*Journal of the Atmospheric Sciences*, vol. 46, no. 20, pp. 3077–3107, 1989. View at Google Scholar · View at Scopus - E. J. Mlawer, S. J. Taubman, P. D. Brown, M. J. Iacono, and S. A. Clough, “Radiative transfer for inhomogeneous atmospheres: RRTM, a validated correlated-k model for the longwave,”
*Journal of Geophysical Research D*, vol. 102, no. 14, pp. 16663–16682, 1997. View at Google Scholar · View at Scopus - K. Brewster, M. Hu, M. Xue, and J. Gao, “Efficient assimilation of radar data at high resolution for short-range numerical weather prediction,” in
*Proceedings of the WWRP International Symposium on Nowcasting and Very Short Range Forecasting*, Toulouse, France, 2005. - R. A. McPherson, C. A. Fiebrich, K. C. Crawford et al., “Statewide monitoring of the mesoscale environment: a technical update on the Oklahoma Mesonet,”
*Journal of Atmospheric and Oceanic Technology*, vol. 24, no. 3, pp. 301–321, 2007. View at Publisher · View at Google Scholar · View at Scopus - D. S. Wilks,
*Statistical Methods in the Atmospheric Sciences*, Academic Press, Boston, Mass, USA, 2nd edition, 2006. - C. E. Hane, R. M. Rabin, T. M. Crawford, H. B. Bluestein, and M. E. Baldwin, “A case study of severe storm development along a dryline within a synoptically active environment. Part II: multiple boundaries and convective initiation,”
*Monthly Weather Review*, vol. 130, no. 4, pp. 900–920, 2002. View at Google Scholar · View at Scopus - R. L. Thompson, R. Edwards, and J. A. Hart, “Evaluation and interpretation of the supercell composite and significant tornado parameters at the Storm Prediction Center,” in
*Proceedings of the 21st Conference on Severe Local Storms*, American Metrological Society, San Antonio, Tex, USA, 2002. - H. E. Brooks, C. A. Doswell III, and J. Cooper, “On the environments of tornadic and nontornadic mesocyclones,”
*Weather & Forecasting*, vol. 9, no. 4, pp. 606–618, 1994. View at Google Scholar · View at Scopus - J. S. Kain, S. J. Weiss, D. R. Bright et al., “Some practical considerations regarding horizontal resolution in the first generation of operational convection-allowing NWP,”
*Weather and Forecasting*, vol. 23, no. 5, pp. 931–952, 2008. View at Publisher · View at Google Scholar · View at Scopus - J. R. Carley, B. R. J. Schwedler, M. E. Baldwin et al., “A proposed model-based methodology for feature-specific prediction for high-impact weather,”
*Weather and Forecasting*, vol. 26, no. 2, pp. 243–249, 2011. View at Publisher · View at Google Scholar · View at Scopus - R. J. Trapp, E. D. Robinson, M. E. Baldwin, N. S. Diffenbaugh, and B. R. J. Schwedler, “Regional climate of hazardous convective weather through high-resolution dynamical downscaling,”
*Climate Dynamics*, vol. 37, no. 3, pp. 677–688, 2011. View at Publisher · View at Google Scholar · View at Scopus - M. L. Miller, V. Lakshmanan, and T. Smith, “An automated method for depicting mesocyclone paths and intensities,”
*Weather and Forecasting*, vol. 28, pp. 570–585, 2013. View at Google Scholar - V. Lakshmanan, T. Smith, G. Stumpf, and K. Hondl, “The warning decision support system-integrated information,”
*Weather and Forecasting*, vol. 22, no. 3, pp. 596–612, 2007. View at Publisher · View at Google Scholar · View at Scopus - G. H. Bryan, J. C. Wyngaard, and J. M. Fritsch, “Resolution requirements for the simulation of deep moist convection,”
*Monthly Weather Review*, vol. 131, pp. 2394–2416, 2003. View at Google Scholar - G. H. Bryan and H. Morrison, “Sensitivity of a simulated squall line to horizontal resolution and parameterization of microphysics,”
*Monthly Weather Review*, vol. 140, no. 1, pp. 202–225, 2012. View at Publisher · View at Google Scholar · View at Scopus - N. Yussouf and D. J. Stensrud, “Comparison of single-parameter and multiparameter ensembles for assimilation of radar observations using the ensemble kalman filter,”
*Monthly Weather Review*, vol. 140, no. 2, pp. 562–586, 2012. View at Publisher · View at Google Scholar · View at Scopus - N. Snook, M. Xue, and Y. Jung, “Analysis of a tornadic mesoscale convective vortex based on ensemble kalman filter assimilation of CASA X-band and WSR-88D radar data,”
*Monthly Weather Review*, vol. 139, no. 11, pp. 3446–3468, 2011. View at Publisher · View at Google Scholar · View at Scopus - N. Snook, M. Xue, and J. Jung, “Ensemble probabilistic forecasts of a tornadic mesoscale convective system from ensemble Kalman filter analyses using WSR-88D and CASA radar data,”
*Monthly Weather Review*, vol. 140, pp. 2126–2146, 2012. View at Google Scholar